2024-03-28T17:45:09.981 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T17:45:09.982 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --archive /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378 --owner mchangir --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} -- /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378/orig.config.yaml 2024-03-28T17:45:10.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T17:45:10.106 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378 branch: wip-mchangir-testing-main-20240318.032620 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: mchangir@redhat.com first_in_suite: false job_id: '7625378' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3645 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 132/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFLluvz0l6rG9k5y5OZMffhZzrCz3kAvkh0E6HBLeoEcNbRADbSD4eg5D+qS4m+n5w1JhHv99prlqs80EQj0rHw= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfr+nHMtoZu1/x36Gu8rwg/COX+iEbxpJ6GLFANG/ybkqNVRDxhWby6DA/W3ywGW9NLxQr9kaSPkWdVCASvSlY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_03:45:23 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T17:45:10.107 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-28T17:45:10.107 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-28T17:45:10.107 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T17:45:10.109 INFO:teuthology.task.internal:Checking packages... 2024-03-28T17:45:10.131 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-28T17:45:10.131 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T17:45:10.132 INFO:teuthology.packaging:ref: None 2024-03-28T17:45:10.132 INFO:teuthology.packaging:tag: None 2024-03-28T17:45:10.132 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-28T17:45:10.132 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:45:10.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-28T17:45:10.354 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-28T17:45:10.355 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T17:45:10.362 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T17:45:10.362 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T17:45:10.414 INFO:teuthology.task.internal:Saving configuration 2024-03-28T17:45:10.429 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T17:45:10.492 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T17:45:10.517 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 17:39:37.467951', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFLluvz0l6rG9k5y5OZMffhZzrCz3kAvkh0E6HBLeoEcNbRADbSD4eg5D+qS4m+n5w1JhHv99prlqs80EQj0rHw='} 2024-03-28T17:45:10.537 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 17:39:37.465989', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfr+nHMtoZu1/x36Gu8rwg/COX+iEbxpJ6GLFANG/ybkqNVRDxhWby6DA/W3ywGW9NLxQr9kaSPkWdVCASvSlY='} 2024-03-28T17:45:10.537 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T17:45:10.543 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-28T17:45:10.543 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-28T17:45:10.544 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T17:45:10.633 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f687d79af70>, signals=[15]) 2024-03-28T17:45:10.633 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T17:45:10.640 INFO:teuthology.task.internal:Opening connections... 2024-03-28T17:45:10.640 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:45:10.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:45:10.746 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:45:10.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:45:10.835 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T17:45:10.841 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-03-28T17:45:10.859 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-03-28T17:45:10.859 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-03-28T17:45:10.917 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-03-28T17:45:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T17:45:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-03-28T17:45:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T17:45:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T17:45:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T17:45:10.918 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-03-28T17:45:10.939 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-03-28T17:45:10.957 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-03-28T17:45:10.957 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-03-28T17:45:11.014 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-03-28T17:45:11.014 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-03-28T17:45:11.014 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-03-28T17:45:11.014 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-03-28T17:45:11.014 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T17:45:11.015 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T17:45:11.015 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-03-28T17:45:11.039 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T17:45:11.048 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T17:45:11.070 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T17:45:11.071 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-03-28T17:45:11.074 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-03-28T17:45:11.090 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T17:45:11.095 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T17:45:11.096 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-03-28T17:45:11.132 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-03-28T17:45:11.285 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T17:45:11.336 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T17:45:11.349 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-28T17:45:11.349 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-28T17:45:11.349 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T17:45:11.349 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T17:45:11.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:45:11.366 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.366 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T17:45:11.366 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-28T17:45:11.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:45:11.367 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.367 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T17:45:11.367 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-03-28T17:45:11.422 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-361.el9.x86_64 2024-03-28T17:45:11.423 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.423 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T17:45:11.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.426 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-361.el9.x86_64 2024-03-28T17:45:11.427 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.427 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T17:45:11.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:11.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2024-03-28T17:45:11.640 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum upgrade -y linux-firmware 2024-03-28T17:45:13.710 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-28T17:45:13.723 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-28T17:45:15.143 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-28T17:45:15.407 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-03-28T17:45:16.309 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-03-28T17:45:18.181 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-03-28T17:45:18.293 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-28T17:45:20.373 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-28T17:45:22.886 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.1 MB 00:00 2024-03-28T17:45:24.611 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-03-28T17:45:25.823 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-28T17:45:31.005 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-28T17:45:31.615 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 807 kB/s | 6.1 MB 00:07 2024-03-28T17:45:33.199 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-03-28T17:45:33.350 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T17:45:33.365 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 1 Package 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 375 M 2024-03-28T17:45:33.366 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-03-28T17:45:34.701 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-28T17:45:39.617 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-28T17:45:41.580 INFO:teuthology.orchestra.run.smithi062.stdout:linux-firmware-20240219-143.el9.noarch.rpm 46 MB/s | 375 MB 00:08 2024-03-28T17:45:41.581 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:45:41.582 INFO:teuthology.orchestra.run.smithi062.stdout:Total 46 MB/s | 375 MB 00:08 2024-03-28T17:45:41.582 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-03-28T17:45:41.664 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-03-28T17:45:41.664 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-03-28T17:45:41.875 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-28T17:45:41.882 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 1 Package 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:45:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 375 M 2024-03-28T17:45:41.884 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-28T17:45:44.165 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-03-28T17:45:44.166 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-03-28T17:45:50.672 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-03-28T17:45:50.926 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:45:51.023 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:45:51.101 INFO:teuthology.orchestra.run.smithi081.stdout:linux-firmware-20240219-143.el9.noarch.rpm 41 MB/s | 375 MB 00:09 2024-03-28T17:45:51.103 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:45:51.103 INFO:teuthology.orchestra.run.smithi081.stdout:Total 41 MB/s | 375 MB 00:09 2024-03-28T17:45:51.103 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-28T17:45:51.196 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-28T17:45:51.196 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-28T17:45:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:45:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:45:52.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:45:52.840 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:45:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-03-28T17:45:52.840 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T17:45:52.841 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:45:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-03-28T17:45:53.149 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T17:45:53.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:45:53.374 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T17:45:53.374 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T17:45:53.376 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2024-03-28T17:45:53.614 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-28T17:45:53.614 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-28T17:46:00.089 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-28T17:46:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:46:00.427 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:46:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:46:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:46:02.067 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:46:02.067 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:46:02.068 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-03-28T17:46:02.068 INFO:teuthology.orchestra.run.smithi081.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T17:46:02.068 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:46:02.068 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-28T17:46:02.348 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T17:46:02.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:46:02.493 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T17:46:02.494 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T17:46:02.496 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi081.front.sepia.ceph.com to complete... 2024-03-28T17:46:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28 17:46:06 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T17:46:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2024-03-28 17:46:06-- 2024-03-28T17:46:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 13s 2024-03-28T17:46:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 321M in 13s (25.0 MB/s) 2024-03-28T17:46:06.580 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T17:46:06.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T17:46:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2024-03-28T17:46:08.674 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2024-03-28T17:46:08.759 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2024-03-28T17:46:16.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28 17:46:16 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T17:46:16.334 INFO:teuthology.orchestra.run.smithi081.stderr:FINISHED --2024-03-28 17:46:16-- 2024-03-28T17:46:16.334 INFO:teuthology.orchestra.run.smithi081.stderr:Total wall clock time: 14s 2024-03-28T17:46:16.335 INFO:teuthology.orchestra.run.smithi081.stderr:Downloaded: 1 files, 321M in 13s (23.8 MB/s) 2024-03-28T17:46:16.337 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T17:46:16.337 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T17:46:16.583 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T17:46:18.276 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying... ######################################## 2024-03-28T17:46:18.331 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing... ######################################## 2024-03-28T17:46:18.411 INFO:teuthology.orchestra.run.smithi081.stdout:Updating / installing... 2024-03-28T17:46:25.964 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T17:46:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T17:46:42.343 INFO:teuthology.orchestra.run.smithi081.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T17:46:55.877 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T17:46:55.877 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T17:46:55.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T17:46:55.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T17:46:55.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T17:46:55.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T17:46:55.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T17:46:55.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T17:46:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T17:46:55.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T17:46:55.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T17:46:55.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T17:46:55.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T17:46:55.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T17:46:56.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T17:46:56.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T17:46:56.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T17:46:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T17:46:56.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T17:46:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T17:46:56.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T17:46:56.007 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T17:46:56.008 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T17:46:56.009 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T17:46:56.010 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T17:46:56.011 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T17:46:56.012 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T17:46:56.013 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T17:46:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T17:46:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T17:46:56.016 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T17:46:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T17:46:56.018 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T17:46:56.019 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T17:46:56.020 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T17:46:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T17:46:56.022 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T17:46:56.023 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T17:46:56.024 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T17:46:56.025 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T17:46:56.026 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T17:46:56.027 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T17:46:56.028 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T17:46:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T17:46:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T17:46:56.031 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T17:46:56.032 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T17:46:56.033 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T17:46:56.034 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T17:46:56.035 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T17:46:56.036 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T17:46:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T17:46:56.038 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T17:46:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T17:46:56.040 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T17:46:56.041 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T17:46:56.042 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T17:46:56.043 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T17:46:56.044 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T17:46:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T17:46:56.046 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T17:46:56.047 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T17:46:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T17:46:56.049 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T17:46:56.050 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T17:46:56.051 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T17:46:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T17:46:56.053 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T17:46:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T17:46:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T17:46:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T17:46:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T17:46:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T17:46:56.059 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T17:46:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T17:46:56.061 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T17:46:56.062 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T17:46:56.063 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T17:46:56.064 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T17:46:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T17:46:56.066 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T17:46:56.067 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T17:46:56.068 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T17:46:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T17:46:56.070 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T17:46:56.071 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T17:46:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T17:46:56.073 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T17:46:56.074 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T17:46:56.075 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T17:46:56.076 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T17:46:56.077 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T17:46:56.078 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T17:46:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T17:46:56.080 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T17:46:56.081 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T17:46:56.082 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T17:46:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T17:46:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T17:46:56.085 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T17:46:56.086 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T17:46:56.087 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T17:46:56.088 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T17:46:56.089 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T17:46:56.090 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T17:46:56.091 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T17:46:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T17:46:56.093 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T17:46:56.094 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T17:46:56.095 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T17:46:56.096 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T17:46:56.097 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T17:46:56.098 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T17:46:56.099 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T17:46:56.100 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T17:46:56.101 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T17:46:56.102 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T17:46:56.103 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T17:46:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T17:46:56.105 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T17:46:56.106 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T17:46:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T17:46:56.108 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T17:46:56.109 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T17:46:56.110 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T17:46:56.111 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T17:46:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T17:46:56.113 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T17:46:56.114 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T17:46:56.115 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:46:56.119 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T17:46:56.119 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T17:46:56.142 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T17:46:56.142 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:46:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:for file in vmlinuz System.map config; do 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: fi 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:done 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T17:46:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-03-28T17:46:56.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T17:46:56.227 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:46:56.228 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T17:46:56.229 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T17:46:56.229 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.8.0-g1df0c7e8a235 2024-03-28T17:46:56.229 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T17:46:56.902 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-03-28T17:46:58.335 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T17:46:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-03-28T17:46:58.367 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-03-28T17:46:58.428 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.39XeJaUtVI 2024-03-28T17:46:58.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.39XeJaUtVI 2024-03-28T17:46:58.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.39XeJaUtVI 2024-03-28T17:46:58.660 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.39XeJaUtVI is 6KB 2024-03-28T17:46:58.710 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.39XeJaUtVI 2024-03-28T17:46:58.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T17:46:58.794 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T17:46:58.794 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T17:46:58.794 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T17:46:58.794 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T17:46:58.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T17:46:58.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-03-28T17:47:04.854 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T17:47:04.854 DEBUG:teuthology.orchestra.run.smithi081:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T17:47:04.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T17:47:04.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T17:47:04.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T17:47:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T17:47:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T17:47:04.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T17:47:04.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T17:47:04.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T17:47:04.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T17:47:04.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T17:47:04.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T17:47:04.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T17:47:04.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T17:47:04.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T17:47:04.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T17:47:04.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T17:47:04.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T17:47:04.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T17:47:04.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T17:47:04.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T17:47:04.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T17:47:04.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T17:47:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T17:47:04.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T17:47:04.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T17:47:04.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T17:47:04.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T17:47:04.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T17:47:04.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T17:47:04.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T17:47:04.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T17:47:04.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T17:47:04.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T17:47:04.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T17:47:04.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T17:47:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T17:47:04.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T17:47:04.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T17:47:04.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T17:47:04.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T17:47:04.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T17:47:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T17:47:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T17:47:04.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T17:47:04.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T17:47:04.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T17:47:04.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T17:47:04.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T17:47:04.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T17:47:04.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T17:47:04.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T17:47:04.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T17:47:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T17:47:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T17:47:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T17:47:04.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T17:47:04.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T17:47:04.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T17:47:04.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T17:47:04.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T17:47:04.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T17:47:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T17:47:04.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T17:47:04.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T17:47:04.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T17:47:04.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T17:47:04.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T17:47:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T17:47:04.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T17:47:04.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T17:47:04.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T17:47:04.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T17:47:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T17:47:05.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T17:47:05.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T17:47:05.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T17:47:05.004 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T17:47:05.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T17:47:05.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T17:47:05.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T17:47:05.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T17:47:05.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T17:47:05.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T17:47:05.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T17:47:05.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T17:47:05.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T17:47:05.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T17:47:05.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T17:47:05.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T17:47:05.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T17:47:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T17:47:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T17:47:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T17:47:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T17:47:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T17:47:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T17:47:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T17:47:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T17:47:05.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T17:47:05.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T17:47:05.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T17:47:05.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T17:47:05.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T17:47:05.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T17:47:05.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T17:47:05.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T17:47:05.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T17:47:05.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T17:47:05.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T17:47:05.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T17:47:05.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T17:47:05.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T17:47:05.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T17:47:05.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T17:47:05.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T17:47:05.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T17:47:05.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T17:47:05.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T17:47:05.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T17:47:05.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T17:47:05.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T17:47:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T17:47:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T17:47:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:47:05.054 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T17:47:05.054 DEBUG:teuthology.orchestra.run.smithi081:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout:for file in vmlinuz System.map config; do 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T17:47:05.076 INFO:teuthology.orchestra.run.smithi081.stdout: fi 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:done 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T17:47:05.077 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-03-28T17:47:05.078 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T17:47:05.158 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:47:05.159 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T17:47:05.163 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T17:47:05.163 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 6.8.0-g1df0c7e8a235 2024-03-28T17:47:05.163 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T17:47:05.760 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-03-28T17:47:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T17:47:07.229 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-03-28T17:47:07.236 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-03-28T17:47:07.252 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.hjmU8TGuxY 2024-03-28T17:47:07.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hjmU8TGuxY 2024-03-28T17:47:07.335 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.hjmU8TGuxY 2024-03-28T17:47:07.475 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.hjmU8TGuxY is 6KB 2024-03-28T17:47:07.524 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.hjmU8TGuxY 2024-03-28T17:47:07.538 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T17:47:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T17:47:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T17:47:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T17:47:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T17:47:07.605 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T17:47:07.748 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-03-28T17:47:28.970 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-28T17:47:28.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:47:28.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:47:37.757 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-28T17:47:37.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:47:37.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:47:47.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-03-28T17:47:56.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-28T17:47:56.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:47:56.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:47:59.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-03-28T17:48:05.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:48:05.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:48:08.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-28T17:48:11.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:48:11.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:48:20.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:48:20.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:48:52.202 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-28T17:48:52.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:48:52.736 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-28T17:48:52.737 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-28T17:48:52.775 INFO:teuthology.orchestra.run.smithi081.stdout:6.8.0-g1df0c7e8a235 2024-03-28T17:48:52.775 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T17:48:52.775 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T17:48:52.775 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T17:48:53.776 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-28T17:48:53.777 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T17:48:53.948 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-03-28T17:48:54.006 DEBUG:teuthology.parallel:result is None 2024-03-28T17:49:11.431 DEBUG:teuthology.orchestra.remote:timed out 2024-03-28T17:49:26.433 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:49:26.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:49:26.886 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-03-28T17:49:27.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:49:27.625 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-28T17:49:27.626 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-03-28T17:49:27.681 INFO:teuthology.orchestra.run.smithi062.stdout:6.8.0-g1df0c7e8a235 2024-03-28T17:49:27.681 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T17:49:27.681 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T17:49:27.681 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T17:49:28.681 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-28T17:49:28.682 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T17:49:28.835 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2024-03-28T17:49:28.904 DEBUG:teuthology.parallel:result is None 2024-03-28T17:49:28.904 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T17:49:28.912 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T17:49:28.912 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T17:49:28.920 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T17:49:28.978 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T17:49:28.989 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T17:49:29.011 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T17:49:29.011 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T17:49:29.019 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T17:49:29.097 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T17:49:29.103 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T17:49:29.104 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T17:49:29.111 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T17:49:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:49:29.217 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:49:29.253 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:49:29.260 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:49:29.264 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T17:49:29.272 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T17:49:29.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T17:49:29.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T17:49:29.374 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T17:49:29.384 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T17:49:29.384 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T17:49:29.391 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T17:49:29.442 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:49:29.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:49:29.664 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:49:29.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:49:29.845 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:49:29.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T17:49:29.948 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:49:30.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:49:30.169 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:49:30.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:49:30.345 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:49:30.345 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T17:49:30.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-03-28T17:49:30.447 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-03-28T17:49:30.573 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T17:49:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T17:49:31.095 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T17:49:31.104 INFO:teuthology.task.internal:Starting timer... 2024-03-28T17:49:31.104 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T17:49:31.157 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T17:49:31.187 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-28T17:49:31.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-03-28T17:49:31.336 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-03-28T17:49:31.340 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-03-28T17:49:31.479 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-03-28T17:49:31.484 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T17:49:31.484 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-03-28T17:49:31.545 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-03-28T17:49:31.545 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-03-28T17:49:31.604 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-03-28T17:49:31.605 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2024-03-28T17:49:31.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T17:49:31.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:49:31.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T17:49:31.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:49:31.780 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T17:49:31.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-03-28T17:49:31.874 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-03-28T17:49:31.976 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T17:49:31.986 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-28T17:49:31.991 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-28T17:49:32.109 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T17:49:32.124 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T17:49:32.125 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T17:55:38.036 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2024-03-28T17:55:38.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:55:38.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:55:38.187 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-03-28T17:55:38.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-28T17:55:38.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:55:38.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:55:38.487 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-28T17:55:38.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-28T17:55:38.643 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T17:55:38.653 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T17:55:38.653 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T17:55:38.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T17:55:38.661 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T17:55:38.661 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T17:55:38.754 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T17:55:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T17:55:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T17:55:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T17:55:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-03-28T17:55:38.921 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-03-28T17:55:38.934 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-03-28T17:55:38.968 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-03-28T17:55:38.983 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T17:55:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T17:55:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T17:55:39.077 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T17:55:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.171 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.201 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-03-28T17:55:39.208 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T17:55:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-03-28T17:55:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:55:39.210 INFO:teuthology.run_tasks:Running task install... 2024-03-28T17:55:39.219 DEBUG:teuthology.task.install:project ceph 2024-03-28T17:55:39.219 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-28T17:55:39.219 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-28T17:55:39.219 INFO:teuthology.task.install:Using flavor: default 2024-03-28T17:55:39.224 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T17:55:39.225 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-28T17:55:39.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T17:55:39.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:55:39.227 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T17:55:39.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:55:39.368 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T17:55:39.368 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T17:55:39.379 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T17:55:39.380 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T17:55:39.482 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T17:55:39.482 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:55:39.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T17:55:39.548 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T17:55:39.548 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:55:39.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T17:55:39.588 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T17:55:39.588 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T17:55:39.647 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T17:55:39.647 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T17:55:39.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T17:55:39.780 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T17:55:39.876 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T17:55:39.912 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T17:55:39.972 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-03-28T17:55:39.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-03-28T17:55:39.999 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-03-28T17:55:40.003 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-03-28T17:55:40.345 INFO:teuthology.orchestra.run.smithi062.stdout:74 files removed 2024-03-28T17:55:40.363 INFO:teuthology.orchestra.run.smithi081.stdout:74 files removed 2024-03-28T17:55:40.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T17:55:40.405 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T17:55:41.348 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-03-28T17:55:41.472 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 148 kB/s | 83 kB 00:00 2024-03-28T17:55:41.849 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-28T17:55:41.997 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-28T17:55:42.308 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-28T17:55:42.499 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-03-28T17:55:42.962 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-28T17:55:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T17:55:44.711 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-03-28T17:55:44.738 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-28T17:55:45.731 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-28T17:55:45.833 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-28T17:55:47.625 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-03-28T17:55:48.257 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-28T17:55:52.375 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-03-28T17:55:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.1 MB 00:00 2024-03-28T17:55:54.435 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-03-28T17:55:54.837 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-28T17:55:56.238 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-28T17:55:57.803 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 21 MB 00:03 2024-03-28T17:56:02.222 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-28T17:56:03.578 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-03-28T17:56:04.987 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:56:04.989 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:56:05.012 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T17:56:05.014 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T17:56:05.015 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T17:56:05.017 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T17:56:05.019 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T17:56:05.023 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T17:56:05.025 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T17:56:05.119 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-03-28T17:56:05.159 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-28T17:56:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T17:56:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T17:56:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-03-28T17:56:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T17:56:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T17:56:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T17:56:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T17:56:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T17:56:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T17:56:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T17:56:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T17:56:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout:Install 150 Packages 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 24 Packages 2024-03-28T17:56:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:56:05.173 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 225 M 2024-03-28T17:56:05.173 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-03-28T17:56:06.157 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:56:06.159 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:56:06.181 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T17:56:06.182 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T17:56:06.183 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T17:56:06.185 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T17:56:06.187 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T17:56:06.191 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T17:56:06.192 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T17:56:06.280 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T17:56:06.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T17:56:06.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T17:56:06.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T17:56:06.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T17:56:06.301 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T17:56:06.302 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T17:56:06.303 INFO:teuthology.orchestra.run.smithi081.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T17:56:06.304 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T17:56:06.305 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T17:56:06.306 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T17:56:06.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T17:56:06.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T17:56:06.309 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T17:56:06.310 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T17:56:06.311 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout:Install 150 Packages 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 24 Packages 2024-03-28T17:56:06.312 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:56:06.313 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 225 M 2024-03-28T17:56:06.313 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-28T17:56:06.465 INFO:teuthology.orchestra.run.smithi062.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-03-28T17:56:06.806 INFO:teuthology.orchestra.run.smithi062.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 6.7 MB/s | 5.4 MB 00:00 2024-03-28T17:56:06.882 INFO:teuthology.orchestra.run.smithi062.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.9 MB/s | 145 kB 00:00 2024-03-28T17:56:07.118 INFO:teuthology.orchestra.run.smithi062.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 9.9 MB/s | 2.3 MB 00:00 2024-03-28T17:56:07.274 INFO:teuthology.orchestra.run.smithi062.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 9.5 MB/s | 1.5 MB 00:00 2024-03-28T17:56:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:(6/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 890 kB 00:00 2024-03-28T17:56:07.691 INFO:teuthology.orchestra.run.smithi062.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-28T17:56:08.280 INFO:teuthology.orchestra.run.smithi081.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 5.5 kB/s | 6.6 kB 00:01 2024-03-28T17:56:08.300 INFO:teuthology.orchestra.run.smithi062.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 8.9 MB/s | 20 MB 00:02 2024-03-28T17:56:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 516 kB/s | 25 kB 00:00 2024-03-28T17:56:08.651 INFO:teuthology.orchestra.run.smithi062.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 10 MB/s | 9.9 MB 00:00 2024-03-28T17:56:08.742 INFO:teuthology.orchestra.run.smithi062.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.4 MB/s | 220 kB 00:00 2024-03-28T17:56:08.779 INFO:teuthology.orchestra.run.smithi081.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.7 MB/s | 890 kB 00:00 2024-03-28T17:56:08.818 INFO:teuthology.orchestra.run.smithi062.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 362 kB/s | 27 kB 00:00 2024-03-28T17:56:08.901 INFO:teuthology.orchestra.run.smithi062.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 380 kB/s | 31 kB 00:00 2024-03-28T17:56:08.921 INFO:teuthology.orchestra.run.smithi081.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.0 MB/s | 145 kB 00:00 2024-03-28T17:56:09.027 INFO:teuthology.orchestra.run.smithi062.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 5.9 MB/s | 746 kB 00:00 2024-03-28T17:56:09.110 INFO:teuthology.orchestra.run.smithi062.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.9 MB/s | 163 kB 00:00 2024-03-28T17:56:09.169 INFO:teuthology.orchestra.run.smithi062.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.2 MB/s | 127 kB 00:00 2024-03-28T17:56:09.243 INFO:teuthology.orchestra.run.smithi081.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-03-28T17:56:09.261 INFO:teuthology.orchestra.run.smithi062.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 5.4 MB/s | 501 kB 00:00 2024-03-28T17:56:09.839 INFO:teuthology.orchestra.run.smithi081.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-03-28T17:56:09.878 INFO:teuthology.orchestra.run.smithi062.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 8.4 MB/s | 5.1 MB 00:00 2024-03-28T17:56:09.936 INFO:teuthology.orchestra.run.smithi062.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 790 kB/s | 45 kB 00:00 2024-03-28T17:56:10.029 INFO:teuthology.orchestra.run.smithi062.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.4 MB/s | 131 kB 00:00 2024-03-28T17:56:10.104 INFO:teuthology.orchestra.run.smithi062.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.1 MB/s | 162 kB 00:00 2024-03-28T17:56:10.187 INFO:teuthology.orchestra.run.smithi062.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.8 MB/s | 321 kB 00:00 2024-03-28T17:56:10.279 INFO:teuthology.orchestra.run.smithi062.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.2 MB/s | 298 kB 00:00 2024-03-28T17:56:10.371 INFO:teuthology.orchestra.run.smithi062.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.1 MB/s | 99 kB 00:00 2024-03-28T17:56:10.430 INFO:teuthology.orchestra.run.smithi062.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.5 MB/s | 85 kB 00:00 2024-03-28T17:56:10.431 INFO:teuthology.orchestra.run.smithi081.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-28T17:56:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:(26/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 8.1 MB/s | 3.1 MB 00:00 2024-03-28T17:56:10.889 INFO:teuthology.orchestra.run.smithi062.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.3 MB/s | 170 kB 00:00 2024-03-28T17:56:10.956 INFO:teuthology.orchestra.run.smithi062.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 516 kB/s | 34 kB 00:00 2024-03-28T17:56:11.048 INFO:teuthology.orchestra.run.smithi062.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.5 MB/s | 139 kB 00:00 2024-03-28T17:56:11.223 INFO:teuthology.orchestra.run.smithi062.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 10 MB/s | 1.8 MB 00:00 2024-03-28T17:56:11.289 INFO:teuthology.orchestra.run.smithi081.stdout:(7/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 4.9 MB/s | 20 MB 00:04 2024-03-28T17:56:11.406 INFO:teuthology.orchestra.run.smithi081.stdout:(8/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-03-28T17:56:11.498 INFO:teuthology.orchestra.run.smithi081.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 278 kB/s | 25 kB 00:00 2024-03-28T17:56:12.024 INFO:teuthology.orchestra.run.smithi062.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 9.3 MB/s | 7.4 MB 00:00 2024-03-28T17:56:12.115 INFO:teuthology.orchestra.run.smithi062.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.7 MB/s | 248 kB 00:00 2024-03-28T17:56:12.191 INFO:teuthology.orchestra.run.smithi062.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 665 kB/s | 49 kB 00:00 2024-03-28T17:56:12.266 INFO:teuthology.orchestra.run.smithi062.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 222 kB/s | 16 kB 00:00 2024-03-28T17:56:12.350 INFO:teuthology.orchestra.run.smithi062.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.1 MB/s | 264 kB 00:00 2024-03-28T17:56:12.508 INFO:teuthology.orchestra.run.smithi062.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-28T17:56:12.700 INFO:teuthology.orchestra.run.smithi062.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 96 kB/s | 18 kB 00:00 2024-03-28T17:56:12.765 INFO:teuthology.orchestra.run.smithi081.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 6.7 MB/s | 9.9 MB 00:01 2024-03-28T17:56:12.840 INFO:teuthology.orchestra.run.smithi081.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 220 kB 00:00 2024-03-28T17:56:12.892 INFO:teuthology.orchestra.run.smithi062.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-28T17:56:12.898 INFO:teuthology.orchestra.run.smithi081.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 469 kB/s | 27 kB 00:00 2024-03-28T17:56:12.957 INFO:teuthology.orchestra.run.smithi081.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 544 kB/s | 31 kB 00:00 2024-03-28T17:56:12.968 INFO:teuthology.orchestra.run.smithi062.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-03-28T17:56:13.009 INFO:teuthology.orchestra.run.smithi062.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 106 kB 00:00 2024-03-28T17:56:13.052 INFO:teuthology.orchestra.run.smithi062.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-03-28T17:56:13.066 INFO:teuthology.orchestra.run.smithi081.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 6.8 MB/s | 746 kB 00:00 2024-03-28T17:56:13.124 INFO:teuthology.orchestra.run.smithi081.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 2.8 MB/s | 163 kB 00:00 2024-03-28T17:56:13.175 INFO:teuthology.orchestra.run.smithi081.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.5 MB/s | 127 kB 00:00 2024-03-28T17:56:13.267 INFO:teuthology.orchestra.run.smithi081.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 5.4 MB/s | 501 kB 00:00 2024-03-28T17:56:13.552 INFO:teuthology.orchestra.run.smithi062.stdout:(42/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 9.2 MB/s | 48 MB 00:05 2024-03-28T17:56:13.577 INFO:teuthology.orchestra.run.smithi062.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 32 kB 00:00 2024-03-28T17:56:13.678 INFO:teuthology.orchestra.run.smithi062.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-28T17:56:13.694 INFO:teuthology.orchestra.run.smithi062.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 173 kB/s | 17 kB 00:00 2024-03-28T17:56:13.720 INFO:teuthology.orchestra.run.smithi062.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 422 kB/s | 10 kB 00:00 2024-03-28T17:56:13.734 INFO:teuthology.orchestra.run.smithi081.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-28T17:56:13.745 INFO:teuthology.orchestra.run.smithi062.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-28T17:56:13.784 INFO:teuthology.orchestra.run.smithi081.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 921 kB/s | 45 kB 00:00 2024-03-28T17:56:13.804 INFO:teuthology.orchestra.run.smithi062.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 44 kB 00:00 2024-03-28T17:56:13.834 INFO:teuthology.orchestra.run.smithi081.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 2.6 MB/s | 131 kB 00:00 2024-03-28T17:56:13.854 INFO:teuthology.orchestra.run.smithi062.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 661 kB 00:00 2024-03-28T17:56:13.885 INFO:teuthology.orchestra.run.smithi081.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 3.2 MB/s | 162 kB 00:00 2024-03-28T17:56:13.896 INFO:teuthology.orchestra.run.smithi062.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-28T17:56:13.921 INFO:teuthology.orchestra.run.smithi062.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 762 kB/s | 46 kB 00:00 2024-03-28T17:56:13.951 INFO:teuthology.orchestra.run.smithi081.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 4.8 MB/s | 321 kB 00:00 2024-03-28T17:56:13.955 INFO:teuthology.orchestra.run.smithi062.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-28T17:56:13.989 INFO:teuthology.orchestra.run.smithi062.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T17:56:14.010 INFO:teuthology.orchestra.run.smithi081.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 5.1 MB/s | 298 kB 00:00 2024-03-28T17:56:14.022 INFO:teuthology.orchestra.run.smithi062.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-28T17:56:14.047 INFO:teuthology.orchestra.run.smithi062.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2024-03-28T17:56:14.060 INFO:teuthology.orchestra.run.smithi081.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 2.0 MB/s | 99 kB 00:00 2024-03-28T17:56:14.102 INFO:teuthology.orchestra.run.smithi081.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 2.0 MB/s | 85 kB 00:00 2024-03-28T17:56:14.198 INFO:teuthology.orchestra.run.smithi062.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-28T17:56:14.240 INFO:teuthology.orchestra.run.smithi062.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-28T17:56:14.298 INFO:teuthology.orchestra.run.smithi062.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-28T17:56:14.386 INFO:teuthology.orchestra.run.smithi081.stdout:(26/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 11 MB/s | 3.1 MB 00:00 2024-03-28T17:56:14.432 INFO:teuthology.orchestra.run.smithi062.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-28T17:56:14.444 INFO:teuthology.orchestra.run.smithi081.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-28T17:56:14.457 INFO:teuthology.orchestra.run.smithi062.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 171 kB/s | 24 kB 00:00 2024-03-28T17:56:14.495 INFO:teuthology.orchestra.run.smithi081.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 693 kB/s | 34 kB 00:00 2024-03-28T17:56:14.516 INFO:teuthology.orchestra.run.smithi062.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 316 kB/s | 18 kB 00:00 2024-03-28T17:56:14.541 INFO:teuthology.orchestra.run.smithi062.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-28T17:56:14.545 INFO:teuthology.orchestra.run.smithi081.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.7 MB/s | 139 kB 00:00 2024-03-28T17:56:14.566 INFO:teuthology.orchestra.run.smithi062.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 657 kB/s | 32 kB 00:00 2024-03-28T17:56:14.700 INFO:teuthology.orchestra.run.smithi062.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-28T17:56:14.725 INFO:teuthology.orchestra.run.smithi062.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-28T17:56:14.745 INFO:teuthology.orchestra.run.smithi081.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 9.0 MB/s | 1.8 MB 00:00 2024-03-28T17:56:14.767 INFO:teuthology.orchestra.run.smithi062.stdout:(66/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-28T17:56:14.842 INFO:teuthology.orchestra.run.smithi062.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2024-03-28T17:56:14.877 INFO:teuthology.orchestra.run.smithi062.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 386 kB/s | 41 kB 00:00 2024-03-28T17:56:14.918 INFO:teuthology.orchestra.run.smithi062.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-28T17:56:14.943 INFO:teuthology.orchestra.run.smithi062.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-28T17:56:14.977 INFO:teuthology.orchestra.run.smithi062.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-28T17:56:15.011 INFO:teuthology.orchestra.run.smithi062.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 735 kB/s | 47 kB 00:00 2024-03-28T17:56:15.036 INFO:teuthology.orchestra.run.smithi062.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 747 kB/s | 43 kB 00:00 2024-03-28T17:56:15.345 INFO:teuthology.orchestra.run.smithi062.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-28T17:56:15.370 INFO:teuthology.orchestra.run.smithi062.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 122 kB/s | 39 kB 00:00 2024-03-28T17:56:15.412 INFO:teuthology.orchestra.run.smithi062.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 41 kB 00:00 2024-03-28T17:56:15.454 INFO:teuthology.orchestra.run.smithi062.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 300 kB 00:00 2024-03-28T17:56:15.479 INFO:teuthology.orchestra.run.smithi062.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 889 kB/s | 56 kB 00:00 2024-03-28T17:56:15.523 INFO:teuthology.orchestra.run.smithi062.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 928 kB/s | 63 kB 00:00 2024-03-28T17:56:15.529 INFO:teuthology.orchestra.run.smithi081.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-03-28T17:56:15.547 INFO:teuthology.orchestra.run.smithi062.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 673 kB/s | 44 kB 00:00 2024-03-28T17:56:15.572 INFO:teuthology.orchestra.run.smithi062.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T17:56:15.606 INFO:teuthology.orchestra.run.smithi062.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-28T17:56:15.664 INFO:teuthology.orchestra.run.smithi062.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-28T17:56:15.690 INFO:teuthology.orchestra.run.smithi062.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-28T17:56:15.750 INFO:teuthology.orchestra.run.smithi062.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-28T17:56:15.846 INFO:teuthology.orchestra.run.smithi081.stdout:(32/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 3.1 MB/s | 17 MB 00:05 2024-03-28T17:56:15.902 INFO:teuthology.orchestra.run.smithi062.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 893 kB/s | 187 kB 00:00 2024-03-28T17:56:15.907 INFO:teuthology.orchestra.run.smithi081.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2145.g3b 663 kB/s | 248 kB 00:00 2024-03-28T17:56:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 599 kB/s | 49 kB 00:00 2024-03-28T17:56:15.955 INFO:teuthology.orchestra.run.smithi081.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 333 kB/s | 16 kB 00:00 2024-03-28T17:56:16.030 INFO:teuthology.orchestra.run.smithi081.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.6 MB/s | 264 kB 00:00 2024-03-28T17:56:16.089 INFO:teuthology.orchestra.run.smithi081.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 318 kB/s | 18 kB 00:00 2024-03-28T17:56:16.156 INFO:teuthology.orchestra.run.smithi081.stdout:(38/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 3.7 MB/s | 748 kB 00:00 2024-03-28T17:56:16.191 INFO:teuthology.orchestra.run.smithi062.stdout:(87/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 1.9 MB/s | 17 MB 00:08 2024-03-28T17:56:16.255 INFO:teuthology.orchestra.run.smithi062.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 373 kB/s | 187 kB 00:00 2024-03-28T17:56:16.333 INFO:teuthology.orchestra.run.smithi062.stdout:(89/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-28T17:56:16.375 INFO:teuthology.orchestra.run.smithi062.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-28T17:56:16.417 INFO:teuthology.orchestra.run.smithi062.stdout:(91/174): libquadmath-11.4.1-3.el9.x86_64.rpm 869 kB/s | 195 kB 00:00 2024-03-28T17:56:16.431 INFO:teuthology.orchestra.run.smithi081.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-03-28T17:56:16.442 INFO:teuthology.orchestra.run.smithi062.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-03-28T17:56:16.476 INFO:teuthology.orchestra.run.smithi062.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 252 kB/s | 25 kB 00:00 2024-03-28T17:56:16.507 INFO:teuthology.orchestra.run.smithi081.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-03-28T17:56:16.549 INFO:teuthology.orchestra.run.smithi081.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 941 kB/s | 106 kB 00:00 2024-03-28T17:56:16.607 INFO:teuthology.orchestra.run.smithi081.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 307 kB 00:00 2024-03-28T17:56:16.624 INFO:teuthology.orchestra.run.smithi081.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 440 kB/s | 32 kB 00:00 2024-03-28T17:56:16.626 INFO:teuthology.orchestra.run.smithi062.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-03-28T17:56:16.710 INFO:teuthology.orchestra.run.smithi062.stdout:(95/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 14 MB/s | 4.1 MB 00:00 2024-03-28T17:56:16.758 INFO:teuthology.orchestra.run.smithi081.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-28T17:56:16.768 INFO:teuthology.orchestra.run.smithi062.stdout:(96/174): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-03-28T17:56:16.775 INFO:teuthology.orchestra.run.smithi081.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 127 kB/s | 17 kB 00:00 2024-03-28T17:56:16.800 INFO:teuthology.orchestra.run.smithi081.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 422 kB/s | 10 kB 00:00 2024-03-28T17:56:16.819 INFO:teuthology.orchestra.run.smithi062.stdout:(97/174): postgresql-private-libs-13.14-1.el9.x 695 kB/s | 133 kB 00:00 2024-03-28T17:56:16.825 INFO:teuthology.orchestra.run.smithi081.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-28T17:56:16.851 INFO:teuthology.orchestra.run.smithi081.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 919 kB/s | 44 kB 00:00 2024-03-28T17:56:16.878 INFO:teuthology.orchestra.run.smithi062.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-28T17:56:16.893 INFO:teuthology.orchestra.run.smithi081.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-28T17:56:16.918 INFO:teuthology.orchestra.run.smithi081.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.3 MB/s | 578 kB 00:00 2024-03-28T17:56:16.944 INFO:teuthology.orchestra.run.smithi081.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.0 MB/s | 46 kB 00:00 2024-03-28T17:56:16.944 INFO:teuthology.orchestra.run.smithi062.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-03-28T17:56:16.969 INFO:teuthology.orchestra.run.smithi081.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-03-28T17:56:17.002 INFO:teuthology.orchestra.run.smithi081.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.3 MB/s | 187 kB 00:00 2024-03-28T17:56:17.011 INFO:teuthology.orchestra.run.smithi062.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-03-28T17:56:17.028 INFO:teuthology.orchestra.run.smithi081.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-03-28T17:56:17.037 INFO:teuthology.orchestra.run.smithi062.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 392 kB/s | 36 kB 00:00 2024-03-28T17:56:17.053 INFO:teuthology.orchestra.run.smithi081.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 717 kB/s | 34 kB 00:00 2024-03-28T17:56:17.095 INFO:teuthology.orchestra.run.smithi062.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-28T17:56:17.121 INFO:teuthology.orchestra.run.smithi062.stdout:(103/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-03-28T17:56:17.186 INFO:teuthology.orchestra.run.smithi081.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-28T17:56:17.229 INFO:teuthology.orchestra.run.smithi081.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-03-28T17:56:17.270 INFO:teuthology.orchestra.run.smithi081.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 78 kB 00:00 2024-03-28T17:56:17.313 INFO:teuthology.orchestra.run.smithi062.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 9.6 MB/s | 5.8 MB 00:00 2024-03-28T17:56:17.338 INFO:teuthology.orchestra.run.smithi062.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 93 kB/s | 22 kB 00:00 2024-03-28T17:56:17.413 INFO:teuthology.orchestra.run.smithi081.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-28T17:56:17.438 INFO:teuthology.orchestra.run.smithi081.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 160 kB/s | 24 kB 00:00 2024-03-28T17:56:17.471 INFO:teuthology.orchestra.run.smithi081.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 564 kB/s | 18 kB 00:00 2024-03-28T17:56:17.497 INFO:teuthology.orchestra.run.smithi081.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-28T17:56:17.522 INFO:teuthology.orchestra.run.smithi081.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 656 kB/s | 32 kB 00:00 2024-03-28T17:56:17.588 INFO:teuthology.orchestra.run.smithi062.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-03-28T17:56:17.647 INFO:teuthology.orchestra.run.smithi081.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-28T17:56:17.672 INFO:teuthology.orchestra.run.smithi081.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-28T17:56:17.714 INFO:teuthology.orchestra.run.smithi081.stdout:(66/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-28T17:56:17.748 INFO:teuthology.orchestra.run.smithi081.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-03-28T17:56:17.773 INFO:teuthology.orchestra.run.smithi081.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 730 kB/s | 41 kB 00:00 2024-03-28T17:56:17.806 INFO:teuthology.orchestra.run.smithi081.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-03-28T17:56:17.832 INFO:teuthology.orchestra.run.smithi081.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-28T17:56:17.866 INFO:teuthology.orchestra.run.smithi081.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-28T17:56:17.891 INFO:teuthology.orchestra.run.smithi081.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 840 kB/s | 47 kB 00:00 2024-03-28T17:56:17.917 INFO:teuthology.orchestra.run.smithi081.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 870 kB/s | 43 kB 00:00 2024-03-28T17:56:18.038 INFO:teuthology.orchestra.run.smithi062.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-03-28T17:56:18.089 INFO:teuthology.orchestra.run.smithi062.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 412 kB/s | 309 kB 00:00 2024-03-28T17:56:18.109 INFO:teuthology.orchestra.run.smithi081.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 219 kB/s | 39 kB 00:00 2024-03-28T17:56:18.206 INFO:teuthology.orchestra.run.smithi062.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-03-28T17:56:18.239 INFO:teuthology.orchestra.run.smithi062.stdout:(110/174): abseil-cpp-20211102.0-3.el9.x86_64.r 592 kB/s | 548 kB 00:00 2024-03-28T17:56:18.326 INFO:teuthology.orchestra.run.smithi081.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-28T17:56:18.346 INFO:teuthology.orchestra.run.smithi062.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 348 kB/s | 49 kB 00:00 2024-03-28T17:56:18.351 INFO:teuthology.orchestra.run.smithi081.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 175 kB/s | 41 kB 00:00 2024-03-28T17:56:18.382 INFO:teuthology.orchestra.run.smithi062.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-03-28T17:56:18.385 INFO:teuthology.orchestra.run.smithi081.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-03-28T17:56:18.410 INFO:teuthology.orchestra.run.smithi081.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-28T17:56:18.435 INFO:teuthology.orchestra.run.smithi081.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-28T17:56:18.460 INFO:teuthology.orchestra.run.smithi081.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 901 kB/s | 44 kB 00:00 2024-03-28T17:56:18.482 INFO:teuthology.orchestra.run.smithi062.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-28T17:56:18.485 INFO:teuthology.orchestra.run.smithi081.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T17:56:18.519 INFO:teuthology.orchestra.run.smithi081.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-28T17:56:18.569 INFO:teuthology.orchestra.run.smithi081.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-28T17:56:18.586 INFO:teuthology.orchestra.run.smithi081.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-28T17:56:18.603 INFO:teuthology.orchestra.run.smithi081.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 3.8 MB/s | 124 kB 00:00 2024-03-28T17:56:18.632 INFO:teuthology.orchestra.run.smithi062.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-03-28T17:56:18.637 INFO:teuthology.orchestra.run.smithi081.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.8 MB/s | 187 kB 00:00 2024-03-28T17:56:18.741 INFO:teuthology.orchestra.run.smithi062.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-03-28T17:56:18.766 INFO:teuthology.orchestra.run.smithi062.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 328 kB/s | 43 kB 00:00 2024-03-28T17:56:18.883 INFO:teuthology.orchestra.run.smithi062.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-03-28T17:56:18.908 INFO:teuthology.orchestra.run.smithi062.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-03-28T17:56:18.942 INFO:teuthology.orchestra.run.smithi062.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 160 kB/s | 32 kB 00:00 2024-03-28T17:56:19.001 INFO:teuthology.orchestra.run.smithi062.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-28T17:56:19.060 INFO:teuthology.orchestra.run.smithi062.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-28T17:56:19.094 INFO:teuthology.orchestra.run.smithi062.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-28T17:56:19.119 INFO:teuthology.orchestra.run.smithi062.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-03-28T17:56:19.154 INFO:teuthology.orchestra.run.smithi081.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 342 kB/s | 187 kB 00:00 2024-03-28T17:56:19.186 INFO:teuthology.orchestra.run.smithi062.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-28T17:56:19.236 INFO:teuthology.orchestra.run.smithi062.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-28T17:56:19.295 INFO:teuthology.orchestra.run.smithi062.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-28T17:56:19.345 INFO:teuthology.orchestra.run.smithi062.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-03-28T17:56:19.354 INFO:teuthology.orchestra.run.smithi081.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 980 kB/s | 195 kB 00:00 2024-03-28T17:56:19.396 INFO:teuthology.orchestra.run.smithi081.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-28T17:56:19.412 INFO:teuthology.orchestra.run.smithi062.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-03-28T17:56:19.446 INFO:teuthology.orchestra.run.smithi062.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-03-28T17:56:19.463 INFO:teuthology.orchestra.run.smithi081.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-28T17:56:19.471 INFO:teuthology.orchestra.run.smithi062.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-03-28T17:56:19.522 INFO:teuthology.orchestra.run.smithi062.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-28T17:56:19.589 INFO:teuthology.orchestra.run.smithi062.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-28T17:56:19.614 INFO:teuthology.orchestra.run.smithi062.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-03-28T17:56:19.639 INFO:teuthology.orchestra.run.smithi062.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2024-03-28T17:56:19.718 INFO:teuthology.orchestra.run.smithi062.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 451 kB/s | 58 kB 00:00 2024-03-28T17:56:19.738 INFO:teuthology.orchestra.run.smithi081.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 63 kB/s | 21 kB 00:00 2024-03-28T17:56:19.748 INFO:teuthology.orchestra.run.smithi062.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-28T17:56:19.780 INFO:teuthology.orchestra.run.smithi081.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 103 kB/s | 25 kB 00:00 2024-03-28T17:56:19.782 INFO:teuthology.orchestra.run.smithi062.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-28T17:56:19.824 INFO:teuthology.orchestra.run.smithi062.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 855 kB/s | 90 kB 00:00 2024-03-28T17:56:19.857 INFO:teuthology.orchestra.run.smithi062.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-28T17:56:19.891 INFO:teuthology.orchestra.run.smithi062.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-03-28T17:56:19.933 INFO:teuthology.orchestra.run.smithi062.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-28T17:56:19.966 INFO:teuthology.orchestra.run.smithi062.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-03-28T17:56:20.000 INFO:teuthology.orchestra.run.smithi062.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-03-28T17:56:20.042 INFO:teuthology.orchestra.run.smithi062.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-03-28T17:56:20.101 INFO:teuthology.orchestra.run.smithi062.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-28T17:56:20.126 INFO:teuthology.orchestra.run.smithi062.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 717 kB/s | 90 kB 00:00 2024-03-28T17:56:20.168 INFO:teuthology.orchestra.run.smithi062.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-28T17:56:20.202 INFO:teuthology.orchestra.run.smithi062.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-03-28T17:56:20.243 INFO:teuthology.orchestra.run.smithi062.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-28T17:56:20.331 INFO:teuthology.orchestra.run.smithi081.stdout:(93/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.0 MB/s | 4.1 MB 00:00 2024-03-28T17:56:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-03-28T17:56:20.377 INFO:teuthology.orchestra.run.smithi062.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-28T17:56:20.506 INFO:teuthology.orchestra.run.smithi081.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-03-28T17:56:20.620 INFO:teuthology.orchestra.run.smithi062.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 9.6 MB/s | 2.3 MB 00:00 2024-03-28T17:56:20.728 INFO:teuthology.orchestra.run.smithi062.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-03-28T17:56:20.778 INFO:teuthology.orchestra.run.smithi062.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 859 kB/s | 42 kB 00:00 2024-03-28T17:56:20.829 INFO:teuthology.orchestra.run.smithi062.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-28T17:56:20.932 INFO:teuthology.orchestra.run.smithi081.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 315 kB/s | 133 kB 00:00 2024-03-28T17:56:20.937 INFO:teuthology.orchestra.run.smithi062.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 666 kB/s | 71 kB 00:00 2024-03-28T17:56:20.988 INFO:teuthology.orchestra.run.smithi062.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 741 kB/s | 36 kB 00:00 2024-03-28T17:56:21.038 INFO:teuthology.orchestra.run.smithi062.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-28T17:56:21.129 INFO:teuthology.orchestra.run.smithi062.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.8 MB/s | 256 kB 00:00 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi062.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 453 kB/s | 23 kB 00:00 2024-03-28T17:56:21.273 INFO:teuthology.orchestra.run.smithi062.stdout:(161/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-03-28T17:56:21.298 INFO:teuthology.orchestra.run.smithi062.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 930 kB/s | 108 kB 00:00 2024-03-28T17:56:21.348 INFO:teuthology.orchestra.run.smithi062.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-28T17:56:21.398 INFO:teuthology.orchestra.run.smithi062.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-03-28T17:56:21.432 INFO:teuthology.orchestra.run.smithi062.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-03-28T17:56:21.432 INFO:teuthology.orchestra.run.smithi081.stdout:(96/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 4.8 MB/s | 48 MB 00:09 2024-03-28T17:56:21.482 INFO:teuthology.orchestra.run.smithi062.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 346 kB/s | 29 kB 00:00 2024-03-28T17:56:21.549 INFO:teuthology.orchestra.run.smithi081.stdout:(97/174): postgresql-contrib-13.14-1.el9.x86_64 749 kB/s | 911 kB 00:01 2024-03-28T17:56:21.599 INFO:teuthology.orchestra.run.smithi062.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 19 MB/s | 2.2 MB 00:00 2024-03-28T17:56:21.658 INFO:teuthology.orchestra.run.smithi062.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.4 MB/s | 1.2 MB 00:00 2024-03-28T17:56:21.728 INFO:teuthology.orchestra.run.smithi062.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.6 MB/s | 465 kB 00:00 2024-03-28T17:56:21.809 INFO:teuthology.orchestra.run.smithi062.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-28T17:56:21.837 INFO:teuthology.orchestra.run.smithi062.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 289 kB/s | 31 kB 00:00 2024-03-28T17:56:21.895 INFO:teuthology.orchestra.run.smithi062.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-28T17:56:21.902 INFO:teuthology.orchestra.run.smithi081.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 611 kB/s | 245 kB 00:00 2024-03-28T17:56:21.929 INFO:teuthology.orchestra.run.smithi062.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 818 kB/s | 74 kB 00:00 2024-03-28T17:56:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 571 kB/s | 248 kB 00:00 2024-03-28T17:56:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:(100/174): postgresql-server-13.14-1.el9.x86_64 4.4 MB/s | 5.8 MB 00:01 2024-03-28T17:56:22.367 INFO:teuthology.orchestra.run.smithi081.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 111 kB/s | 36 kB 00:00 2024-03-28T17:56:22.400 INFO:teuthology.orchestra.run.smithi081.stdout:(102/174): qatlib-24.02.0-1.el9.x86_64.rpm 445 kB/s | 221 kB 00:00 2024-03-28T17:56:22.617 INFO:teuthology.orchestra.run.smithi081.stdout:(103/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 124 kB/s | 47 kB 00:00 2024-03-28T17:56:22.676 INFO:teuthology.orchestra.run.smithi081.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 82 kB/s | 22 kB 00:00 2024-03-28T17:56:22.768 INFO:teuthology.orchestra.run.smithi081.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 234 kB/s | 94 kB 00:00 2024-03-28T17:56:23.001 INFO:teuthology.orchestra.run.smithi081.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-03-28T17:56:23.268 INFO:teuthology.orchestra.run.smithi081.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-28T17:56:23.310 INFO:teuthology.orchestra.run.smithi081.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 571 kB/s | 309 kB 00:00 2024-03-28T17:56:23.343 INFO:teuthology.orchestra.run.smithi081.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-03-28T17:56:23.436 INFO:teuthology.orchestra.run.smithi081.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-03-28T17:56:23.461 INFO:teuthology.orchestra.run.smithi081.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2024-03-28T17:56:23.553 INFO:teuthology.orchestra.run.smithi081.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 580 kB/s | 67 kB 00:00 2024-03-28T17:56:23.578 INFO:teuthology.orchestra.run.smithi081.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-28T17:56:23.737 INFO:teuthology.orchestra.run.smithi081.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-28T17:56:23.820 INFO:teuthology.orchestra.run.smithi081.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-03-28T17:56:23.845 INFO:teuthology.orchestra.run.smithi081.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-03-28T17:56:23.937 INFO:teuthology.orchestra.run.smithi081.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-03-28T17:56:23.971 INFO:teuthology.orchestra.run.smithi081.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-03-28T17:56:24.071 INFO:teuthology.orchestra.run.smithi081.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-03-28T17:56:24.105 INFO:teuthology.orchestra.run.smithi081.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-03-28T17:56:24.138 INFO:teuthology.orchestra.run.smithi081.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-03-28T17:56:24.205 INFO:teuthology.orchestra.run.smithi081.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-03-28T17:56:24.239 INFO:teuthology.orchestra.run.smithi081.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-03-28T17:56:24.273 INFO:teuthology.orchestra.run.smithi081.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-28T17:56:24.348 INFO:teuthology.orchestra.run.smithi081.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-28T17:56:24.382 INFO:teuthology.orchestra.run.smithi081.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-28T17:56:24.462 INFO:teuthology.orchestra.run.smithi062.stdout:(174/174): librados2-19.0.0-2145.g3be9a5c9.el9. 816 kB/s | 3.4 MB 00:04 2024-03-28T17:56:24.465 INFO:teuthology.orchestra.run.smithi081.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-03-28T17:56:24.477 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:56:24.477 INFO:teuthology.orchestra.run.smithi062.stdout:Total 12 MB/s | 225 MB 00:19 2024-03-28T17:56:24.490 INFO:teuthology.orchestra.run.smithi081.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-03-28T17:56:24.516 INFO:teuthology.orchestra.run.smithi081.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-03-28T17:56:24.595 INFO:teuthology.orchestra.run.smithi081.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 151 kB/s | 19 kB 00:00 2024-03-28T17:56:24.625 INFO:teuthology.orchestra.run.smithi081.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-03-28T17:56:24.683 INFO:teuthology.orchestra.run.smithi081.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-03-28T17:56:24.709 INFO:teuthology.orchestra.run.smithi081.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 411 kB/s | 46 kB 00:00 2024-03-28T17:56:24.734 INFO:teuthology.orchestra.run.smithi081.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2024-03-28T17:56:24.784 INFO:teuthology.orchestra.run.smithi081.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-03-28T17:56:24.843 INFO:teuthology.orchestra.run.smithi081.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-28T17:56:24.868 INFO:teuthology.orchestra.run.smithi081.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-03-28T17:56:24.893 INFO:teuthology.orchestra.run.smithi081.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-03-28T17:56:24.944 INFO:teuthology.orchestra.run.smithi081.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-03-28T17:56:24.969 INFO:teuthology.orchestra.run.smithi081.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-03-28T17:56:24.994 INFO:teuthology.orchestra.run.smithi081.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-28T17:56:25.044 INFO:teuthology.orchestra.run.smithi081.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 592 kB/s | 59 kB 00:00 2024-03-28T17:56:25.070 INFO:teuthology.orchestra.run.smithi081.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-03-28T17:56:25.095 INFO:teuthology.orchestra.run.smithi081.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-03-28T17:56:25.179 INFO:teuthology.orchestra.run.smithi081.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-28T17:56:25.204 INFO:teuthology.orchestra.run.smithi081.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 673 kB/s | 90 kB 00:00 2024-03-28T17:56:25.238 INFO:teuthology.orchestra.run.smithi081.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-28T17:56:25.279 INFO:teuthology.orchestra.run.smithi081.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-03-28T17:56:25.346 INFO:teuthology.orchestra.run.smithi081.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-28T17:56:25.422 INFO:teuthology.orchestra.run.smithi081.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-03-28T17:56:25.489 INFO:teuthology.orchestra.run.smithi081.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2024-03-28T17:56:25.939 INFO:teuthology.orchestra.run.smithi081.stdout:(152/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.3 MB/s | 3.1 MB 00:00 2024-03-28T17:56:26.156 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-03-28T17:56:26.181 INFO:teuthology.orchestra.run.smithi081.stdout:(153/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.8 MB/s | 3.4 MB 00:00 2024-03-28T17:56:26.384 INFO:teuthology.orchestra.run.smithi081.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-03-28T17:56:26.515 INFO:teuthology.orchestra.run.smithi081.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.9 MB/s | 248 kB 00:00 2024-03-28T17:56:26.532 INFO:teuthology.orchestra.run.smithi081.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 120 kB/s | 42 kB 00:00 2024-03-28T17:56:26.557 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-03-28T17:56:26.557 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-03-28T17:56:26.649 INFO:teuthology.orchestra.run.smithi081.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 537 kB/s | 71 kB 00:00 2024-03-28T17:56:26.666 INFO:teuthology.orchestra.run.smithi081.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 274 kB/s | 36 kB 00:00 2024-03-28T17:56:26.783 INFO:teuthology.orchestra.run.smithi081.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2024-03-28T17:56:26.908 INFO:teuthology.orchestra.run.smithi081.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 185 kB/s | 23 kB 00:00 2024-03-28T17:56:26.992 INFO:teuthology.orchestra.run.smithi081.stdout:(161/174): NetworkManager-libnm-1.47.2-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-03-28T17:56:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 812 kB/s | 108 kB 00:00 2024-03-28T17:56:27.126 INFO:teuthology.orchestra.run.smithi081.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 556 kB/s | 256 kB 00:00 2024-03-28T17:56:27.152 INFO:teuthology.orchestra.run.smithi081.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 979 kB/s | 155 kB 00:00 2024-03-28T17:56:27.186 INFO:teuthology.orchestra.run.smithi081.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 955 kB/s | 136 kB 00:00 2024-03-28T17:56:27.286 INFO:teuthology.orchestra.run.smithi081.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 403 kB/s | 64 kB 00:00 2024-03-28T17:56:27.311 INFO:teuthology.orchestra.run.smithi081.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 181 kB/s | 29 kB 00:00 2024-03-28T17:56:27.378 INFO:teuthology.orchestra.run.smithi081.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.3 MB/s | 1.2 MB 00:00 2024-03-28T17:56:27.520 INFO:teuthology.orchestra.run.smithi081.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.2 MB/s | 465 kB 00:00 2024-03-28T17:56:27.645 INFO:teuthology.orchestra.run.smithi081.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 250 kB/s | 31 kB 00:00 2024-03-28T17:56:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.7 MB/s | 2.3 MB 00:00 2024-03-28T17:56:27.787 INFO:teuthology.orchestra.run.smithi081.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-03-28T17:56:27.904 INFO:teuthology.orchestra.run.smithi081.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 2.2 MB 00:00 2024-03-28T17:56:28.062 INFO:teuthology.orchestra.run.smithi081.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 212 kB/s | 74 kB 00:00 2024-03-28T17:56:28.073 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:56:28.074 INFO:teuthology.orchestra.run.smithi081.stdout:Total 10 MB/s | 225 MB 00:21 2024-03-28T17:56:29.077 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-03-28T17:56:29.078 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-03-28T17:56:29.697 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-28T17:56:30.074 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-28T17:56:30.075 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-28T17:56:32.567 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-28T17:56:32.567 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-28T17:56:37.027 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-03-28T17:56:37.163 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T17:56:37.206 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T17:56:37.240 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:56:37.283 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:56:37.372 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T17:56:37.473 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T17:56:37.532 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T17:56:37.563 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T17:56:37.653 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T17:56:38.088 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T17:56:38.180 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:56:38.341 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:56:38.371 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:56:38.501 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:56:38.562 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:56:38.647 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T17:56:38.964 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T17:56:39.027 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T17:56:39.052 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:56:39.217 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:56:39.253 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:56:39.344 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:56:39.398 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T17:56:39.438 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:56:39.491 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:56:39.811 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:56:39.865 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:56:39.930 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T17:56:40.028 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-28T17:56:40.120 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T17:56:40.267 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T17:56:40.275 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T17:56:40.415 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:56:40.459 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:56:40.470 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:56:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T17:56:40.551 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:56:40.653 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T17:56:40.706 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T17:56:40.739 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T17:56:40.745 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:40.807 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:40.827 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T17:56:41.162 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:41.246 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T17:56:41.304 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:56:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T17:56:41.455 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:56:41.480 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:56:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T17:56:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T17:56:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T17:56:41.599 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:56:41.645 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:56:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T17:56:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T17:56:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T17:56:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T17:56:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T17:56:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T17:56:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T17:56:42.063 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T17:56:42.096 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:56:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T17:56:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T17:56:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T17:56:42.249 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:56:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T17:56:42.281 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:56:42.363 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:56:42.408 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T17:56:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T17:56:42.441 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:56:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T17:56:42.501 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:56:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T17:56:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T17:56:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T17:56:42.751 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:56:42.754 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T17:56:42.804 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:56:42.849 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T17:56:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T17:56:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T17:56:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T17:56:43.159 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T17:56:43.359 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:56:43.439 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:56:43.631 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:43.686 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:43.848 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T17:56:43.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T17:56:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:56:44.062 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T17:56:44.130 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:56:44.160 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:56:44.203 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T17:56:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T17:56:44.271 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T17:56:44.323 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T17:56:44.349 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T17:56:44.371 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T17:56:44.384 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T17:56:44.422 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T17:56:44.488 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T17:56:44.599 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T17:56:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T17:56:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T17:56:44.796 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T17:56:44.798 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T17:56:44.839 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T17:56:44.899 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T17:56:44.943 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T17:56:44.984 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T17:56:45.022 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T17:56:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T17:56:45.121 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T17:56:45.139 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T17:56:45.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T17:56:45.204 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T17:56:45.246 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T17:56:45.365 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T17:56:45.415 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T17:56:45.423 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T17:56:45.498 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T17:56:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T17:56:45.579 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T17:56:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T17:56:45.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T17:56:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T17:56:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T17:56:45.752 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T17:56:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T17:56:45.888 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T17:56:45.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T17:56:46.029 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T17:56:46.235 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T17:56:46.292 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T17:56:46.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T17:56:46.390 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T17:56:46.455 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T17:56:46.523 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T17:56:46.548 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T17:56:46.570 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T17:56:46.616 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T17:56:46.653 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T17:56:46.673 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T17:56:46.686 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T17:56:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:56:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:56:46.805 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T17:56:46.862 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T17:56:46.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T17:56:46.915 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T17:56:46.957 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T17:56:47.052 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T17:56:47.187 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T17:56:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T17:56:47.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T17:56:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T17:56:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T17:56:47.570 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T17:56:47.657 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T17:56:47.665 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T17:56:47.726 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T17:56:47.731 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T17:56:47.909 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T17:56:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T17:56:47.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T17:56:48.006 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T17:56:48.009 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T17:56:48.044 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T17:56:48.085 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T17:56:48.088 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T17:56:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T17:56:48.189 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T17:56:48.190 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T17:56:48.226 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T17:56:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T17:56:48.265 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T17:56:48.306 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T17:56:48.333 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T17:56:48.373 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T17:56:48.434 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T17:56:48.439 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T17:56:48.484 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T17:56:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T17:56:48.519 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T17:56:48.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T17:56:48.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T17:56:48.682 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T17:56:48.738 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T17:56:48.795 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T17:56:48.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T17:56:48.852 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T17:56:48.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T17:56:48.977 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T17:56:48.989 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T17:56:49.031 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T17:56:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T17:56:49.078 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T17:56:49.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T17:56:49.116 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T17:56:49.149 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T17:56:49.245 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T17:56:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T17:56:49.313 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T17:56:49.470 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T17:56:49.605 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T17:56:49.659 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T17:56:49.730 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T17:56:49.804 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T17:56:50.049 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T17:56:50.105 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T17:56:50.274 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T17:56:50.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T17:56:50.320 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T17:56:50.359 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T17:56:50.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T17:56:50.398 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T17:56:50.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T17:56:50.438 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T17:56:50.470 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T17:56:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T17:56:50.543 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T17:56:50.584 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T17:56:50.635 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T17:56:50.661 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T17:56:50.686 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T17:56:50.811 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T17:56:50.848 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T17:56:50.884 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T17:56:50.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T17:56:50.937 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T17:56:50.974 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T17:56:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T17:56:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T17:56:51.132 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T17:56:51.241 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T17:56:51.301 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T17:56:51.336 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T17:56:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T17:56:51.531 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T17:56:51.594 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T17:56:51.997 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T17:56:52.141 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T17:56:52.546 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T17:56:52.602 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T17:56:52.635 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T17:56:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T17:56:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T17:56:52.836 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T17:56:53.063 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T17:56:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T17:56:53.196 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T17:56:53.748 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T17:56:53.837 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T17:56:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T17:56:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T17:56:53.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T17:56:53.981 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T17:56:54.120 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T17:56:54.357 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T17:56:54.424 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:56:54.546 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:56:54.576 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T17:56:54.635 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T17:56:55.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T17:56:55.606 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:56:55.660 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T17:56:55.690 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:56:55.716 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T17:56:55.754 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T17:56:55.792 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T17:56:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T17:56:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T17:56:56.246 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T17:56:56.278 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:56:56.399 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:56:56.434 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T17:56:56.489 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T17:56:56.999 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T17:56:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:56:57.105 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:56:57.133 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T17:56:58.497 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:56:58.537 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:56:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:56:59.241 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:56:59.398 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:56:59.398 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T17:56:59.398 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T17:56:59.398 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:56:59.441 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:57:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:57:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:57:00.804 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:57:00.851 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:57:01.015 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:57:01.015 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T17:57:01.015 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T17:57:01.015 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:01.060 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-03-28T17:57:12.908 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-03-28T17:57:12.909 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-03-28T17:57:12.909 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:13.147 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:57:13.194 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:57:13.195 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:13.195 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T17:57:13.195 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-03-28T17:57:13.827 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:57:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:57:14.132 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:14.132 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T17:57:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:14.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:57:14.954 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:57:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T17:57:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:57:14.954 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:57:14.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:15.208 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:57:15.281 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:15.769 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:57:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:57:15.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:57:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T17:57:16.094 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:57:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:57:16.160 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:16.161 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T17:57:16.161 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:57:16.161 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:57:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:16.678 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:57:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:57:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:57:16.762 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:57:16.762 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:16.763 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T17:57:16.763 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:57:16.763 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:57:16.763 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T17:57:16.880 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T17:57:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T17:57:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:57:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:57:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:57:17.517 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:57:17.556 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:57:17.608 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T17:57:17.708 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T17:57:17.735 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:57:18.065 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:57:18.111 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:57:18.506 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:57:18.541 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:57:18.692 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:57:18.880 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T17:57:18.918 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:57:18.992 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:57:18.992 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:18.992 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T17:57:18.992 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:57:18.992 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:57:18.993 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:19.053 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:57:19.093 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:57:19.184 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T17:57:19.225 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:57:19.253 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:57:19.393 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:57:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T17:57:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:57:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:19.764 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:57:19.802 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:57:19.898 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T17:57:20.228 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T17:57:20.263 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:57:20.355 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:20.397 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:57:20.478 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:57:20.478 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:20.479 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T17:57:20.479 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:20.584 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:57:20.665 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:20.893 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T17:57:20.925 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:57:20.998 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:21.051 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:57:21.065 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:57:21.123 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:57:21.123 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:21.123 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T17:57:21.123 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:57:21.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:21.201 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:57:21.278 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:21.635 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:57:21.710 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:57:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T17:57:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T17:57:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T17:57:26.502 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:57:26.557 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T17:57:26.592 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T17:57:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T17:57:26.630 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T17:57:26.760 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T17:57:26.822 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T17:57:26.883 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T17:57:26.894 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T17:57:26.954 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T17:57:27.008 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T17:57:27.019 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T17:57:27.043 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T17:57:27.067 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T17:57:27.087 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T17:57:27.120 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T17:57:27.165 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T17:57:27.183 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T17:57:27.229 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T17:57:27.263 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T17:57:27.277 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T17:57:27.300 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T17:57:27.352 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T17:57:27.390 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T17:57:27.430 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T17:57:27.458 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T17:57:27.561 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T17:57:27.593 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T17:57:28.357 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:28.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:28.436 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:28.471 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:28.490 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:57:29.863 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T17:57:29.948 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T17:57:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T17:57:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T17:57:30.049 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T17:57:30.103 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T17:57:30.158 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T17:57:30.198 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T17:57:30.231 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T17:57:30.263 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T17:57:30.265 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T17:57:30.306 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T17:57:30.307 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:57:30.318 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T17:57:30.352 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T17:57:30.387 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T17:57:30.430 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T17:57:30.430 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:57:30.444 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:57:30.486 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T17:57:30.487 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:57:30.574 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:57:30.616 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:57:30.623 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T17:57:30.624 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:57:30.660 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T17:57:30.707 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T17:57:30.740 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T17:57:30.762 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:57:30.772 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T17:57:30.806 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T17:57:30.841 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T17:57:30.852 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T17:57:30.893 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T17:57:30.921 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T17:57:30.951 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T17:57:30.985 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T17:57:31.019 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T17:57:31.052 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T17:57:31.054 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T17:57:31.086 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:57:31.119 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T17:57:31.142 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T17:57:31.154 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.155 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T17:57:31.222 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T17:57:31.280 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T17:57:31.323 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:57:31.356 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T17:57:31.394 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.394 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.484 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.485 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:57:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:57:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:57:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T17:57:31.722 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:57:31.723 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:57:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T17:57:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T17:57:31.845 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:57:31.865 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:57:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:57:31.914 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T17:57:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T17:57:32.010 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T17:57:32.134 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:57:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:57:32.149 INFO:teuthology.orchestra.run.smithi081.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T17:57:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:32.409 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:57:32.410 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T17:57:32.410 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T17:57:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T17:57:37.135 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T17:57:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T17:57:37.137 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T17:57:37.138 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T17:57:37.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T17:57:37.140 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T17:57:37.141 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T17:57:37.142 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T17:57:37.143 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T17:57:37.144 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T17:57:37.145 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T17:57:37.146 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T17:57:37.147 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:57:37.391 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T17:57:37.392 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T17:57:37.408 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T17:57:37.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T17:57:37.410 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T17:57:37.411 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T17:57:37.412 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T17:57:37.413 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T17:57:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T17:57:37.415 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T17:57:37.416 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T17:57:37.417 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T17:57:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T17:57:37.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T17:57:37.420 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:57:37.420 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T17:57:37.420 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T17:57:37.420 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T17:57:39.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T17:57:39.676 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:39.676 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-03-28T17:57:39.676 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.677 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.678 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.679 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: jq-1.6-16.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T17:57:39.680 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T17:57:39.681 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T17:57:39.682 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T17:57:39.683 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T17:57:39.684 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T17:57:39.685 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T17:57:39.686 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T17:57:39.687 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T17:57:39.688 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T17:57:39.688 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:39.688 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:40.179 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.180 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.182 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-16.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T17:57:40.183 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T17:57:40.184 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T17:57:40.185 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T17:57:40.186 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T17:57:40.187 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T17:57:40.188 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T17:57:40.189 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T17:57:40.190 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T17:57:40.191 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T17:57:40.191 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T17:57:40.191 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T17:57:40.191 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:40.191 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-03-28T17:57:40.356 DEBUG:teuthology.parallel:result is None 2024-03-28T17:57:40.881 DEBUG:teuthology.parallel:result is None 2024-03-28T17:57:40.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:57:41.126 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T17:57:41.183 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:57:41.184 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:57:41.184 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T17:57:41.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:57:41.318 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T17:57:41.384 INFO:teuthology.orchestra.run.smithi081.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:57:41.384 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:57:41.384 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T17:57:41.386 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T17:57:41.386 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:41.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T17:57:41.477 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:41.477 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T17:57:41.558 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T17:57:41.559 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:41.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T17:57:41.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T17:57:41.748 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:41.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T17:57:41.824 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T17:57:41.941 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T17:57:41.941 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:41.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T17:57:42.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T17:57:42.142 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:42.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T17:57:42.220 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T17:57:42.336 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T17:57:42.337 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:42.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T17:57:42.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T17:57:42.529 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:42.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T17:57:42.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T17:57:42.712 INFO:teuthology.run_tasks:Running task ceph... 2024-03-28T17:57:42.789 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-28T17:57:42.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-03-28T17:57:42.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2024-03-28T17:57:42.865 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-28T17:57:42.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T17:57:42.869 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T17:57:42.939 INFO:tasks.ceph:Creating extra log directories... 2024-03-28T17:57:42.939 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T17:57:42.943 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T17:57:43.017 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-28T17:57:43.017 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-28T17:57:43.017 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-28T17:57:43.017 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:43.017 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T17:57:43.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T17:57:43.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T17:57:43.339 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:43.340 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T17:57:43.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T17:57:43.593 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T17:57:43.670 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-28T17:57:43.670 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-28T17:57:43.670 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625378', 'branch': 'wip-mchangir-testing-main-20240318.032620', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7625378', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing-main-20240318.032620'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing-main-20240318.032620', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, 'owner': 'mchangir', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3645, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'sleep_before_teardown': 0, 'subset': '132/256', 'suite': 'fs', 'suite_branch': 'wip-mchangir-testing-main-20240318.032620', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFLluvz0l6rG9k5y5OZMffhZzrCz3kAvkh0E6HBLeoEcNbRADbSD4eg5D+qS4m+n5w1JhHv99prlqs80EQj0rHw=', 'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfr+nHMtoZu1/x36Gu8rwg/COX+iEbxpJ6GLFANG/ybkqNVRDxhWby6DA/W3ywGW9NLxQr9kaSPkWdVCASvSlY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-27_03:45:23', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-28T17:57:43.671 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T17:57:43.674 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T17:57:43.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T17:57:43.753 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T17:57:43.825 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:43.825 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T17:57:43.863 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T17:57:43.863 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-03-28T17:57:43.940 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-28 17:57:32.897413726 +0000 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-28 17:55:05.852064580 +0000 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-28 17:55:05.852064580 +0000 2024-03-28T17:57:43.941 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-28 17:55:05.852064580 +0000 2024-03-28T17:57:43.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T17:57:44.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-28T17:57:44.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-28T17:57:44.045 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000333611 s, 1.5 MB/s 2024-03-28T17:57:44.048 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T17:57:44.087 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:44.167 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-28 17:57:32.898413711 +0000 2024-03-28T17:57:44.168 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-28 17:55:06.440055409 +0000 2024-03-28T17:57:44.168 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-28 17:55:06.440055409 +0000 2024-03-28T17:57:44.168 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-28 17:55:06.440055409 +0000 2024-03-28T17:57:44.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T17:57:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-28T17:57:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-28T17:57:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000327571 s, 1.6 MB/s 2024-03-28T17:57:44.289 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T17:57:44.343 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-03-28T17:57:44.388 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T17:57:44.388 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-28 17:57:32.898413711 +0000 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-28 17:55:07.032046176 +0000 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-28 17:55:07.032046176 +0000 2024-03-28T17:57:44.389 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-28 17:55:07.032046176 +0000 2024-03-28T17:57:44.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T17:57:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-28T17:57:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-28T17:57:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000336514 s, 1.5 MB/s 2024-03-28T17:57:44.497 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T17:57:44.536 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-28 17:57:32.899413695 +0000 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-28 17:55:07.614037099 +0000 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-28 17:55:07.614037099 +0000 2024-03-28T17:57:44.617 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-28 17:55:07.614037099 +0000 2024-03-28T17:57:44.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T17:57:44.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-28T17:57:44.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-28T17:57:44.723 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000279675 s, 1.8 MB/s 2024-03-28T17:57:44.726 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T17:57:44.764 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:44.764 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:44.764 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T17:57:44.813 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T17:57:44.814 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-28 17:57:32.582332893 +0000 2024-03-28T17:57:44.903 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-28 17:55:02.419428102 +0000 2024-03-28T17:57:44.904 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-28 17:55:02.419428102 +0000 2024-03-28T17:57:44.904 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-28 17:55:02.419428102 +0000 2024-03-28T17:57:44.904 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T17:57:44.976 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-28T17:57:44.977 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-28T17:57:44.977 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000316029 s, 1.6 MB/s 2024-03-28T17:57:44.979 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T17:57:45.019 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-28 17:57:32.583332874 +0000 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-28 17:55:03.007417028 +0000 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-28 17:55:03.007417028 +0000 2024-03-28T17:57:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-28 17:55:03.007417028 +0000 2024-03-28T17:57:45.110 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T17:57:45.188 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-28T17:57:45.188 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-28T17:57:45.188 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000335142 s, 1.5 MB/s 2024-03-28T17:57:45.192 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T17:57:45.245 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-03-28T17:57:45.294 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-28 17:57:32.583332874 +0000 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-28 17:55:03.587406105 +0000 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-28 17:55:03.587406105 +0000 2024-03-28T17:57:45.295 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-28 17:55:03.587406105 +0000 2024-03-28T17:57:45.295 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T17:57:45.381 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-28T17:57:45.381 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-28T17:57:45.381 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000349297 s, 1.5 MB/s 2024-03-28T17:57:45.386 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T17:57:45.441 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-28 17:57:32.584332855 +0000 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-28 17:55:04.182394900 +0000 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-28 17:55:04.182394900 +0000 2024-03-28T17:57:45.491 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-28 17:55:04.182394900 +0000 2024-03-28T17:57:45.492 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T17:57:45.580 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-28T17:57:45.580 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-28T17:57:45.580 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000329774 s, 1.6 MB/s 2024-03-28T17:57:45.584 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T17:57:45.638 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:45.638 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T17:57:45.638 INFO:tasks.ceph:Generating config... 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] debug client = 20 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] debug ms = 1 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] pid file = 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-28T17:57:45.640 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-28T17:57:45.640 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-28T17:57:45.641 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-28T17:57:45.642 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-28T17:57:45.642 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-28T17:57:45.642 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-28T17:57:45.643 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-28T17:57:45.643 INFO:tasks.ceph:Setting up mon.a... 2024-03-28T17:57:45.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-28T17:57:45.826 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.keyring 2024-03-28T17:57:45.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-28T17:57:45.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T17:57:46.056 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.62'), ('mon.b', '172.21.15.81'), ('mon.c', '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]')] 2024-03-28T17:57:46.056 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.62,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-28T17:57:46.057 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:46.057 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T17:57:46.107 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.62 --add b 172.21.15.81 --addv c '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: generated fsid aedb2108-ca9d-4a87-ad07-9f4b969c6f45 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 0 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:fsid aedb2108-ca9d-4a87-ad07-9f4b969c6f45 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:last_changed 2024-03-28T17:57:46.267099+0000 2024-03-28T17:57:46.266 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-03-28T17:57:46.267099+0000 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:min_mon_release 17 (quincy) 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:election_strategy: 1 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:2: [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] mon.c 2024-03-28T17:57:46.267 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-28T17:57:46.270 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T17:57:46.320 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aedb2108-ca9d-4a87-ad07-9f4b969c6f45... 2024-03-28T17:57:46.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T17:57:46.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T17:57:46.495 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-03-28T17:57:46.495 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2024-03-28T17:57:46.495 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T17:57:46.495 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.496 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T17:57:46.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.62,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]" 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = aedb2108-ca9d-4a87-ad07-9f4b969c6f45 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2024-03-28T17:57:46.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.499 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2024-03-28T17:57:46.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-03-28T17:57:46.501 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T17:57:46.502 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T17:57:46.503 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2024-03-28T17:57:46.504 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = True 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = True 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-03-28T17:57:46.505 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.506 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.507 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.62,172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790]" 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = aedb2108-ca9d-4a87-ad07-9f4b969c6f45 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-03-28T17:57:46.508 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug trim objects = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.509 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2024-03-28T17:57:46.510 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-03-28T17:57:46.511 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T17:57:46.512 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-03-28T17:57:46.513 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-03-28T17:57:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds balancer = 20 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-03-28T17:57:46.515 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-03-28T17:57:46.544 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-28T17:57:46.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-28T17:57:46.694 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-28T17:57:46.695 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:46.695 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-28T17:57:46.749 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:46.749 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-28T17:57:46.799 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:57:46.800 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:46.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T17:57:46.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T17:57:46.956 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:57:46.956 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:47.055 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:57:47.055 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:47.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T17:57:47.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T17:57:47.207 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:57:47.207 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:47.300 INFO:tasks.ceph:Setting up mon nodes... 2024-03-28T17:57:47.300 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-28T17:57:47.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T17:57:47.497 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T17:57:47.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T17:57:47.702 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T17:57:47.713 INFO:tasks.ceph:Setting up mds nodes... 2024-03-28T17:57:47.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T17:57:47.898 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T17:57:47.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-28T17:57:48.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-28T17:57:48.189 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-28T17:57:48.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-28T17:57:48.298 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T17:57:48.485 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T17:57:48.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-28T17:57:48.592 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-28T17:57:48.765 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-28T17:57:48.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-28T17:57:48.863 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-28T17:57:48.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T17:57:48.994 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-28T17:57:49.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-28T17:57:49.184 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-28T17:57:49.246 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-28T17:57:49.373 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-28T17:57:49.423 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-28T17:57:49.557 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-28T17:57:49.615 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-28T17:57:49.615 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T17:57:49.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-28T17:57:49.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:49.721 INFO:tasks.ceph:role: osd.0 2024-03-28T17:57:49.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:57:49.721 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T17:57:49.824 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:57:49.824 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:57:49.824 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:57:49.824 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:57:49.825 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:57:50.341 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-03-28T17:57:50.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-03-28T17:57:50.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-28T17:57:50.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-28T17:57:50.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-28T17:57:50.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:50.742 INFO:tasks.ceph:role: osd.1 2024-03-28T17:57:50.742 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:57:50.742 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:57:50.845 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:57:50.846 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:57:51.339 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-03-28T17:57:51.346 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-03-28T17:57:51.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-28T17:57:51.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-28T17:57:51.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-28T17:57:51.687 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:51.687 INFO:tasks.ceph:role: osd.2 2024-03-28T17:57:51.688 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:57:51.688 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T17:57:51.785 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:57:51.786 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:57:52.295 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-03-28T17:57:52.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-03-28T17:57:52.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-28T17:57:52.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-28T17:57:52.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-28T17:57:52.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T17:57:52.703 INFO:tasks.ceph:role: osd.3 2024-03-28T17:57:52.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:57:52.703 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T17:57:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:57:52.809 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:57:52.809 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:57:52.809 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:57:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:57:52.810 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:57:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:57:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:57:52.810 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:57:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:57:53.280 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-03-28T17:57:53.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-03-28T17:57:53.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-28T17:57:53.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-28T17:57:53.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:53.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.689+0000 7f5efaa13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-28T17:57:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.690+0000 7f5efaa13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-28T17:57:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.690+0000 7f5efaa13640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T17:57:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.690+0000 7f5efaa13640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T17:57:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.690+0000 7f5efaa13640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T17:57:53.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:53.699+0000 7f5efaa13640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-28T17:57:56.626 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-28T17:57:56.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:57:56.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.871+0000 7f2d4161b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-28T17:57:56.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.872+0000 7f2d4161b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-28T17:57:56.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.872+0000 7f2d4161b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T17:57:56.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.872+0000 7f2d4161b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T17:57:56.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.872+0000 7f2d4161b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T17:57:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:57:56.874+0000 7f2d4161b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-28T17:57:59.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-28T17:57:59.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:00.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.038+0000 7f187d4c8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-28T17:58:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.039+0000 7f187d4c8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-28T17:58:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.039+0000 7f187d4c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T17:58:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.039+0000 7f187d4c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T17:58:00.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.039+0000 7f187d4c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T17:58:00.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:00.041+0000 7f187d4c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-28T17:58:02.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-28T17:58:03.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:03.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.228+0000 7fba97e1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-28T17:58:03.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.228+0000 7fba97e1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-28T17:58:03.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.228+0000 7fba97e1d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T17:58:03.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.228+0000 7fba97e1d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T17:58:03.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.228+0000 7fba97e1d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T17:58:03.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:03.231+0000 7fba97e1d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-28T17:58:06.414 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-28T17:58:06.513 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-28T17:58:06.613 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T17:58:06.613 INFO:tasks.ceph:role: osd.4 2024-03-28T17:58:06.614 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:58:06.614 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T17:58:06.713 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:58:06.714 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:58:07.247 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-28T17:58:07.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-28T17:58:07.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-28T17:58:07.441 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-28T17:58:07.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-28T17:58:07.629 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T17:58:07.629 INFO:tasks.ceph:role: osd.5 2024-03-28T17:58:07.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:58:07.629 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:58:07.718 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:58:08.204 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-28T17:58:08.209 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-28T17:58:08.209 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-28T17:58:08.344 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-28T17:58:08.435 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-28T17:58:08.514 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T17:58:08.514 INFO:tasks.ceph:role: osd.6 2024-03-28T17:58:08.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:58:08.515 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T17:58:08.593 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:58:08.593 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:58:08.594 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:58:09.112 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-28T17:58:09.118 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-28T17:58:09.118 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-28T17:58:09.297 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-28T17:58:09.393 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-28T17:58:09.483 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T17:58:09.483 INFO:tasks.ceph:role: osd.7 2024-03-28T17:58:09.483 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:58:09.483 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T17:58:09.579 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T17:58:09.579 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T17:58:09.579 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T17:58:09.580 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T17:58:10.043 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-03-28T17:58:10.049 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-03-28T17:58:10.049 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-28T17:58:10.179 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-28T17:58:10.269 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:10.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.410+0000 7f44976d9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-28T17:58:10.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.413+0000 7f44976d9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-28T17:58:10.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.413+0000 7f44976d9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T17:58:10.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.413+0000 7f44976d9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T17:58:10.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.413+0000 7f44976d9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T17:58:10.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:10.418+0000 7f44976d9640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-28T17:58:13.595 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-28T17:58:13.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:58:13.680 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:58:13.692 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:13.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.854+0000 7f7f58b8f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-28T17:58:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.854+0000 7f7f58b8f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-28T17:58:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.854+0000 7f7f58b8f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T17:58:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.855+0000 7f7f58b8f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T17:58:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.855+0000 7f7f58b8f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T17:58:13.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:13.857+0000 7f7f58b8f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-28T17:58:16.784 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-28T17:58:16.923 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:17.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.061+0000 7fd31e40d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-28T17:58:17.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.061+0000 7fd31e40d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-28T17:58:17.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.061+0000 7fd31e40d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T17:58:17.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.061+0000 7fd31e40d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T17:58:17.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.061+0000 7fd31e40d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T17:58:17.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:17.064+0000 7fd31e40d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-28T17:58:19.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-28T17:58:20.080 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.219+0000 7f84f521a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-28T17:58:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.220+0000 7f84f521a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-28T17:58:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.220+0000 7f84f521a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T17:58:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.220+0000 7f84f521a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T17:58:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.220+0000 7f84f521a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T17:58:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:20.222+0000 7f84f521a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-28T17:58:23.136 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-28T17:58:23.231 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-28T17:58:23.231 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-28T17:58:23.340 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-28T17:58:23.442 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-28T17:58:23.539 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.539 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-28T17:58:23.640 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-28T17:58:23.743 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-28T17:58:23.837 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:23.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-28T17:58:23.939 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:23.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-28T17:58:24.040 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.040 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-28T17:58:24.134 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-28T17:58:24.228 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.228 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-28T17:58:24.323 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-28T17:58:24.416 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.416 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-28T17:58:24.552 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.552 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-28T17:58:24.642 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:24.642 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-28T17:58:24.702 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:24.702 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-28T17:58:24.759 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.759 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-28T17:58:24.816 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:24.816 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-28T17:58:24.867 INFO:tasks.ceph:Adding keys to all mons... 2024-03-28T17:58:24.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T17:58:24.875 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T17:58:24.960 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.y] 2024-03-28T17:58:24.960 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCbrwVmoaC4HRAAkXQCXYk32KN+luxT8Vq4Fw== 2024-03-28T17:58:24.960 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-03-28T17:58:24.960 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCbrwVmiN6lNRAASRu6FTHQ/HbZ9UMk5eRuGA== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCcrwVmOwlWCxAAy4KMrmuYYqT2SfSiGD66Qg== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQChrwVmZUQhKRAANCc7Q6HOlBbK5mE3MdsUxA== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCkrwVmieT7MxAAA4Jy/TUwA8b8Aw8T8jJWUA== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCorwVmFCZuAhAAA8Ijw5rMdzAhLPPkGiX5Aw== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCrrwVm8HOoDRAAOxX4XAx0/PCKL3kDqHufVw== 2024-03-28T17:58:24.961 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCbrwVmkaDuKRAAwxEKu9XTzWl7KntUs79Npg== 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCcrwVmnTb+HBAA4yirM35LUHfjsGj6/1LvwQ== 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCcrwVmVtGuLRAARp0kxBW1fLsrcY7d98TeRA== 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCyrwVm1V2WGBAAZQvGgbJYWczcHwiMkLSRPA== 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQC1rwVm5hgEMxAACg9Yuu+HjS/BKeea9szDjA== 2024-03-28T17:58:24.962 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.6] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQC5rwVm792+AxAARa5j8feGpeKwSbGop8RWqQ== 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.7] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQC8rwVm6gEeDRAAlC30ODXpPSrfSowRFQmFIA== 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCcrwVm9FlXOxAAfVJXlLgfSVrvwPjHrZ1V6Q== 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout:[client.2] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCdrwVmfZcWCxAAnEDmPwLZf3n/I2datkH3vA== 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout:[client.1] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCdrwVmAatNFhAAZk3tH45tdhLWLMq+ofHs5w== 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout:[client.3] 2024-03-28T17:58:24.963 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCdrwVmJ35SIRAAPt9Sl3SquQDeZ0hxYWTrUA== 2024-03-28T17:58:24.966 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-03-28T17:58:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCbrwVmoaC4HRAAkXQCXYk32KN+luxT8Vq4Fw== 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCbrwVmiN6lNRAASRu6FTHQ/HbZ9UMk5eRuGA== 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCcrwVmOwlWCxAAy4KMrmuYYqT2SfSiGD66Qg== 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQChrwVmZUQhKRAANCc7Q6HOlBbK5mE3MdsUxA== 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCkrwVmieT7MxAAA4Jy/TUwA8b8Aw8T8jJWUA== 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-03-28T17:58:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCorwVmFCZuAhAAA8Ijw5rMdzAhLPPkGiX5Aw== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCrrwVm8HOoDRAAOxX4XAx0/PCKL3kDqHufVw== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCbrwVmkaDuKRAAwxEKu9XTzWl7KntUs79Npg== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCcrwVmnTb+HBAA4yirM35LUHfjsGj6/1LvwQ== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCcrwVmVtGuLRAARp0kxBW1fLsrcY7d98TeRA== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCyrwVm1V2WGBAAZQvGgbJYWczcHwiMkLSRPA== 2024-03-28T17:58:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC1rwVm5hgEMxAACg9Yuu+HjS/BKeea9szDjA== 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC5rwVm792+AxAARa5j8feGpeKwSbGop8RWqQ== 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC8rwVm6gEeDRAAlC30ODXpPSrfSowRFQmFIA== 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCcrwVm9FlXOxAAfVJXlLgfSVrvwPjHrZ1V6Q== 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCdrwVmfZcWCxAAnEDmPwLZf3n/I2datkH3vA== 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-03-28T17:58:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCdrwVmAatNFhAAZk3tH45tdhLWLMq+ofHs5w== 2024-03-28T17:58:24.970 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2024-03-28T17:58:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCdrwVmJ35SIRAAPt9Sl3SquQDeZ0hxYWTrUA== 2024-03-28T17:58:24.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T17:58:24.979 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T17:58:25.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:25.161 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:25.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:25.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:25.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.769 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:25.920 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T17:58:26.072 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T17:58:26.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:26.225 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:26.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:26.381 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T17:58:26.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.531 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.829 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:26.976 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T17:58:27.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.276 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.438 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.590 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T17:58:27.736 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-28T17:58:27.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-28T17:58:27.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T17:58:27.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-28T17:58:28.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-28T17:58:28.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T17:58:28.344 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-28T17:58:28.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-28T17:58:28.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T17:58:28.669 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-28T17:58:28.745 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:28.752 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T17:58:28.803 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-28T17:58:28.803 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-28T17:58:28.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-28T17:58:28.808 INFO:tasks.ceph.mon.a:Started 2024-03-28T17:58:28.808 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-28T17:58:28.809 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-28T17:58:28.814 INFO:tasks.ceph.mon.b:Started 2024-03-28T17:58:28.814 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-28T17:58:28.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-28T17:58:28.821 INFO:tasks.ceph.mon.c:Started 2024-03-28T17:58:28.821 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-28T17:58:28.821 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-28T17:58:28.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-28T17:58:28.826 INFO:tasks.ceph.mgr.y:Started 2024-03-28T17:58:28.826 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-28T17:58:28.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-28T17:58:28.831 INFO:tasks.ceph.mgr.x:Started 2024-03-28T17:58:28.831 INFO:tasks.ceph:Setting crush tunables to default 2024-03-28T17:58:28.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd crush tunables default 2024-03-28T17:58:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.010+0000 7f3df4a00640 1 Processor -- start 2024-03-28T17:58:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.010+0000 7f3df4a00640 1 -- start start 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3df00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3df4a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 0x7f3df006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3df4a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df00b4240 con 0x7f3df0145400 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3df4a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df00cef40 con 0x7f3df00cd3d0 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3df4a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00cf170 con 0x7f3df006dd70 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3def400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3df00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3defe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 0x7f3df006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3deea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3df0145400 0x7f3df00cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38080/0 (socket says 172.21.15.62:38080) 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3deea00640 1 -- 172.21.15.62:0/3954963624 learned_addr learned my addr 172.21.15.62:0/3954963624 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:29.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3defe00640 1 -- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 msgr2=0x7f3df006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3defe00640 1 -- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 msgr2=0x7f3df006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3defe00640 1 --2- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 0x7f3df006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3def400640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 msgr2=0x7f3df00b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3def400640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 msgr2=0x7f3df00b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3def400640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3df00b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3defe00640 1 --2- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 0x7f3df006c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:58:29.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.012+0000 7f3def400640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3df00b49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:58:29.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.021+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855836954 0 0) 0x7f3ddc001110 con 0x7f3df0145400 2024-03-28T17:58:29.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.021+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd4003520 con 0x7f3df0145400 2024-03-28T17:58:29.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1616276478 0 0) 0x7f3dd4003520 con 0x7f3df0145400 2024-03-28T17:58:29.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 msgr2=0x7f3df00b49e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T17:58:29.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3df00b49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 msgr2=0x7f3df006c980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 --2- 172.21.15.62:0/3954963624 >> v2:172.21.15.81:3300/0 conn(0x7f3df00cd3d0 0x7f3df006c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df00cf410 con 0x7f3df0145400 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f3ddc001840 con 0x7f3df0145400 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 >> v1:172.21.15.62:6789/0 conn(0x7f3df0145400 legacy=0x7f3df00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3dd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00cf170 con 0x7f3dd40039b0 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd400b9f0 con 0x7f3dd4004800 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd400bbf0 con 0x7f3df006dd70 2024-03-28T17:58:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3deea00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.022+0000 7f3defe00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3dd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.023+0000 7f3deea00640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 msgr2=0x7f3dd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.023+0000 7f3deea00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df006dd70 0x7f3dd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.023+0000 7f3deea00640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd40039b0 msgr2=0x7f3dd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.023+0000 7f3deea00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.023+0000 7f3deea00640 1 -- 172.21.15.62:0/3954963624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4003520 con 0x7f3dd4004800 2024-03-28T17:58:29.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.024+0000 7f3deea00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c490 tx=0x7f3dd4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd006f02daf27585 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.024+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00c0e0 con 0x7f3dd4004800 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.024+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc030070 con 0x7f3dd4004800 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.024+0000 7f3dee000640 1 -- 172.21.15.62:0/3954963624 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0058d0 con 0x7f3dd4004800 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.025+0000 7f3df4a00640 1 -- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dd4004800 msgr2=0x7f3dd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.025+0000 7f3df4a00640 1 --2- 172.21.15.62:0/3954963624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c490 tx=0x7f3dd4005830 comp rx=0 tx=0).stop 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.025+0000 7f3defe00640 1 -- 172.21.15.62:0/3954963624 reap_dead start 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.025+0000 7f3df4a00640 1 -- 172.21.15.62:0/3954963624 shutdown_connections 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.025+0000 7f3df4a00640 1 -- 172.21.15.62:0/3954963624 >> 172.21.15.62:0/3954963624 conn(0x7f3df0074370 msgr2=0x7f3df0066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.026+0000 7f3df4a00640 1 -- 172.21.15.62:0/3954963624 shutdown_connections 2024-03-28T17:58:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.026+0000 7f3df4a00640 1 -- 172.21.15.62:0/3954963624 wait complete. 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.027+0000 7f3df4a00640 1 Processor -- start 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.027+0000 7f3df4a00640 1 -- start start 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.028+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.028+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 0x7f3df015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.028+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df015b610 0x7f3df015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3df4a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00b31e0 con 0x7f3dd4004800 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3df4a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00b44c0 con 0x7f3df015b1b0 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3df4a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df006e150 con 0x7f3df015b610 2024-03-28T17:58:29.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52726/0 (socket says 172.21.15.62:52726) 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 -- 172.21.15.62:0/1495569444 learned_addr learned my addr 172.21.15.62:0/1495569444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3deea00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 0x7f3df015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3defe00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df015b610 0x7f3df015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 -- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df015b610 msgr2=0x7f3df015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df015b610 0x7f3df015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 -- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 msgr2=0x7f3df015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 0x7f3df015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3def400640 1 -- 172.21.15.62:0/1495569444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df00ca440 con 0x7f3dd4004800 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.029+0000 7f3deea00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 0x7f3df015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3def400640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3de000d800 tx=0x7f3de000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:29.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0018020 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de000b6c0 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df00ca720 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000bab0 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df00cace0 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.030+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3de001b020 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.031+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3de001d530 con 0x7f3dd4004800 2024-03-28T17:58:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.031+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df004f400 con 0x7f3dd4004800 2024-03-28T17:58:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.034+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3de0014410 con 0x7f3dd4004800 2024-03-28T17:58:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.108+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3df0070970 con 0x7f3dd4004800 2024-03-28T17:58:29.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.111+0000 7f3de7400640 1 -- 172.21.15.62:0/1495569444 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3de00059f0 con 0x7f3dd4004800 2024-03-28T17:58:29.111 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 msgr2=0x7f3df00c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3de000d800 tx=0x7f3de000dcc0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 shutdown_connections 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3df015b610 0x7f3df015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015b1b0 0x7f3df015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 --2- 172.21.15.62:0/1495569444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd4004800 0x7f3df00c8660 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.112+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 >> 172.21.15.62:0/1495569444 conn(0x7f3df0074370 msgr2=0x7f3df0144390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.113+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 shutdown_connections 2024-03-28T17:58:29.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:29.113+0000 7f3df4a00640 1 -- 172.21.15.62:0/1495569444 wait complete. 2024-03-28T17:58:29.130 INFO:tasks.ceph:check_enable_crimson: False 2024-03-28T17:58:29.131 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-28T17:58:29.131 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:29.131 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-28T17:58:29.219 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:29.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-28T17:58:29.293 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:29.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-28T17:58:29.422 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:58:29.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-28T17:58:29.512 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:29.512 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-28T17:58:29.599 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:29.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-28T17:58:29.730 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:29.730 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-28T17:58:29.811 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:29.811+0000 7fa523c45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:58:29.813 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:29.813+0000 7f4054643040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:58:29.857 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:58:29.857 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-28T17:58:29.880 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:29.881+0000 7fa523c45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:58:29.883 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:29.883+0000 7f4054643040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:58:29.966 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 33eb1999-42d3-4888-8067-853d79e67f47 0 2024-03-28T17:58:30.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.153+0000 7faf43200640 1 Processor -- start 2024-03-28T17:58:30.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.153+0000 7faf43200640 1 -- start start 2024-03-28T17:58:30.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf43200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf3c0d36d0 0x7faf3c0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf43200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0d2630 con 0x7faf3c15df40 2024-03-28T17:58:30.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf43200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0d2910 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf43200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0d3240 con 0x7faf3c0d36d0 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf41400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faf3c0d0990 0x7faf3c0b6cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57112/0 (socket says 172.21.15.81:57112) 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf41400640 1 -- 172.21.15.81:0/795348427 learned_addr learned my addr 172.21.15.81:0/795348427 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf42800640 1 --2- 172.21.15.81:0/795348427 >> v2:172.21.15.62:3300/0 conn(0x7faf3c0d36d0 0x7faf3c0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927253082 0 0) 0x7faf3c0d2910 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.155+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003520 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043950751 0 0) 0x7faf24003520 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 >> v2:172.21.15.62:3300/0 conn(0x7faf3c0d36d0 msgr2=0x7faf3c0b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 --2- 172.21.15.81:0/795348427 >> v2:172.21.15.62:3300/0 conn(0x7faf3c0d36d0 0x7faf3c0b9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 >> v1:172.21.15.62:6789/0 conn(0x7faf3c15df40 legacy=0x7faf3c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0d2bf0 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7faf2c001670 con 0x7faf3c0d0990 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 >> v1:172.21.15.81:6789/0 conn(0x7faf3c0d0990 legacy=0x7faf3c0b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 0x7faf24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf24004800 0x7faf24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf240051c0 0x7faf2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0d3240 con 0x7faf240051c0 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400c0c0 con 0x7faf24003a00 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400c2c0 con 0x7faf24004800 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf41e00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 0x7faf24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf41400640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf24004800 0x7faf24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.156+0000 7faf42800640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf240051c0 0x7faf2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf41e00640 1 -- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf24004800 msgr2=0x7faf24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf41e00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf24004800 0x7faf24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf41e00640 1 -- 172.21.15.81:0/795348427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf240051c0 msgr2=0x7faf2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf41e00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf240051c0 0x7faf2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf41e00640 1 -- 172.21.15.81:0/795348427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24003520 con 0x7faf24003a00 2024-03-28T17:58:30.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.157+0000 7faf42800640 1 -- 172.21.15.81:0/795348427 reap_dead start 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf41e00640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 0x7faf24003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faf30001750 tx=0x7faf24005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3db5b14fd077cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30001370 con 0x7faf24003a00 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300052d0 con 0x7faf24003a00 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf40a00640 1 -- 172.21.15.81:0/795348427 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30003830 con 0x7faf24003a00 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf43200640 1 -- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 msgr2=0x7faf24003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.158+0000 7faf43200640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 0x7faf24003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faf30001750 tx=0x7faf24005830 comp rx=0 tx=0).stop 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.159+0000 7faf43200640 1 -- 172.21.15.81:0/795348427 shutdown_connections 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.159+0000 7faf43200640 1 --2- 172.21.15.81:0/795348427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24003a00 0x7faf24003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.159+0000 7faf43200640 1 -- 172.21.15.81:0/795348427 >> 172.21.15.81:0/795348427 conn(0x7faf3c071cf0 msgr2=0x7faf3c066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.159+0000 7faf43200640 1 -- 172.21.15.81:0/795348427 shutdown_connections 2024-03-28T17:58:30.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.159+0000 7faf43200640 1 -- 172.21.15.81:0/795348427 wait complete. 2024-03-28T17:58:30.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.160+0000 7faf43200640 1 Processor -- start 2024-03-28T17:58:30.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.160+0000 7faf43200640 1 -- start start 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf24003a00 0x7faf3c150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 0x7faf3c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0b98b0 con 0x7faf24003a00 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0b9730 con 0x7faf3c159a30 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.161+0000 7faf43200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0b9a30 con 0x7faf3c1f43b0 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf41400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf41400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44762/0 (socket says 172.21.15.81:44762) 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf41400640 1 -- 172.21.15.81:0/1374818965 learned_addr learned my addr 172.21.15.81:0/1374818965 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 0x7faf3c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf41e00640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf24003a00 0x7faf3c150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 -- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 msgr2=0x7faf3c1f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 -- 172.21.15.81:0/1374818965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf24003a00 msgr2=0x7faf3c150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf24003a00 0x7faf3c150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 -- 172.21.15.81:0/1374818965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0b3750 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf41400640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.162+0000 7faf42800640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 0x7faf3c0b4510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0d4be0 tx=0x7faf380115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38011db0 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3801b070 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0b3a30 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38003990 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c159f80 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faf38003b30 con 0x7faf3c1f43b0 2024-03-28T17:58:30.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.163+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7faf380164c0 con 0x7faf3c1f43b0 2024-03-28T17:58:30.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.164+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c04f400 con 0x7faf3c1f43b0 2024-03-28T17:58:30.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.167+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faf38004030 con 0x7faf3c1f43b0 2024-03-28T17:58:30.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.245+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "33eb1999-42d3-4888-8067-853d79e67f47", "id": 0} v 0) v1 -- 0x7faf3c0c8af0 con 0x7faf3c1f43b0 2024-03-28T17:58:30.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.251+0000 7faf36000640 1 -- 172.21.15.81:0/1374818965 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33eb1999-42d3-4888-8067-853d79e67f47", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf38004210 con 0x7faf3c1f43b0 2024-03-28T17:58:30.250 INFO:teuthology.orchestra.run.smithi081.stdout:0 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 msgr2=0x7faf3c0b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 0x7faf3c0b4510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0d4be0 tx=0x7faf380115a0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 shutdown_connections 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf3c1f43b0 0x7faf3c0b4510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c159a30 0x7faf3c1f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 --2- 172.21.15.81:0/1374818965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf24003a00 0x7faf3c150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 >> 172.21.15.81:0/1374818965 conn(0x7faf3c071cf0 msgr2=0x7faf3c0c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 shutdown_connections 2024-03-28T17:58:30.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.252+0000 7faf43200640 1 -- 172.21.15.81:0/1374818965 wait complete. 2024-03-28T17:58:30.267 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new e7677661-2399-419b-af43-51c30814f65f 1 2024-03-28T17:58:30.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.439+0000 7f059b200640 1 Processor -- start 2024-03-28T17:58:30.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.439+0000 7f059b200640 1 -- start start 2024-03-28T17:58:30.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059b200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f059c15df40 0x7f059c0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d0990 0x7f059c0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059b200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0d2630 con 0x7f059c0d36d0 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059b200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0d2400 con 0x7f059c15df40 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0d3240 con 0x7f059c0d0990 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f059c15df40 0x7f059c0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f059a800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d0990 0x7f059c0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f059c15df40 0x7f059c0b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44770/0 (socket says 172.21.15.81:44770) 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 -- 172.21.15.81:0/1082621476 learned_addr learned my addr 172.21.15.81:0/1082621476 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 -- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d0990 msgr2=0x7f059c0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 --2- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d0990 0x7f059c0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 -- 172.21.15.81:0/1082621476 >> v1:172.21.15.81:6789/0 conn(0x7f059c0d36d0 legacy=0x7f059c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.441+0000 7f0599e00640 1 -- 172.21.15.81:0/1082621476 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0d2910 con 0x7f059c15df40 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.443+0000 7f0599e00640 1 --2- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c0b9730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f059c0d33c0 tx=0x7f058c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32dae717b788cc67 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.443+0000 7f0598a00640 1 -- 172.21.15.81:0/1082621476 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c034070 con 0x7f059c15df40 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.443+0000 7f0598a00640 1 -- 172.21.15.81:0/1082621476 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c030840 con 0x7f059c15df40 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.443+0000 7f0598a00640 1 -- 172.21.15.81:0/1082621476 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c030c10 con 0x7f059c15df40 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 -- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 msgr2=0x7f059c0b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 --2- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c0b9730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f059c0d33c0 tx=0x7f058c02b980 comp rx=0 tx=0).stop 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 -- 172.21.15.81:0/1082621476 shutdown_connections 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 --2- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d0990 0x7f059c0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 --2- 172.21.15.81:0/1082621476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c0b9730 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 -- 172.21.15.81:0/1082621476 >> 172.21.15.81:0/1082621476 conn(0x7f059c071cf0 msgr2=0x7f059c066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 -- 172.21.15.81:0/1082621476 shutdown_connections 2024-03-28T17:58:30.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.444+0000 7f059b200640 1 -- 172.21.15.81:0/1082621476 wait complete. 2024-03-28T17:58:30.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.445+0000 7f059b200640 1 Processor -- start 2024-03-28T17:58:30.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.446+0000 7f059b200640 1 -- start start 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 0x7f059c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 0x7f059c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0b6cd0 con 0x7f059c0d0990 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0d2ed0 con 0x7f059c15df40 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0b6e50 con 0x7f059c0d36d0 2024-03-28T17:58:30.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 0x7f059c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44782/0 (socket says 172.21.15.81:44782) 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059a800640 1 -- 172.21.15.81:0/2932116719 learned_addr learned my addr 172.21.15.81:0/2932116719 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599e00640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 0x7f059c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 -- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 msgr2=0x7f059c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 -- 172.21.15.81:0/2932116719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 msgr2=0x7f059c14fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 0x7f059c14fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599400640 1 -- 172.21.15.81:0/2932116719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0ce430 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f0599e00640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 0x7f059c14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.447+0000 7f059a800640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0599400640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 0x7f059c15e320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f058800c340 tx=0x7f058800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880052d0 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588005490 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0cac70 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588007260 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0ce9b0 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0588007400 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.448+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0588003aa0 con 0x7f059c0d36d0 2024-03-28T17:58:30.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.449+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c04f400 con 0x7f059c0d36d0 2024-03-28T17:58:30.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.451+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0588003ca0 con 0x7f059c0d36d0 2024-03-28T17:58:30.512 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.513+0000 7fa523c45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:58:30.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.531+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e7677661-2399-419b-af43-51c30814f65f", "id": 1} v 0) v1 -- 0x7f059c06b820 con 0x7f059c0d36d0 2024-03-28T17:58:30.532 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.532+0000 7f4054643040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:58:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.537+0000 7f0592000640 1 -- 172.21.15.81:0/2932116719 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7677661-2399-419b-af43-51c30814f65f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f05880035f0 con 0x7f059c0d36d0 2024-03-28T17:58:30.537 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 msgr2=0x7f059c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 0x7f059c15e320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f058800c340 tx=0x7f058800c800 comp rx=0 tx=0).stop 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 shutdown_connections 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c15df40 0x7f059c15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f059c0d36d0 0x7f059c15e320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 --2- 172.21.15.81:0/2932116719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c0d0990 0x7f059c14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 >> 172.21.15.81:0/2932116719 conn(0x7f059c071cf0 msgr2=0x7f059c0c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 shutdown_connections 2024-03-28T17:58:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.539+0000 7f059b200640 1 -- 172.21.15.81:0/2932116719 wait complete. 2024-03-28T17:58:30.561 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 584b7a16-2729-4cf0-a52a-2384928197c9 2 2024-03-28T17:58:30.587 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.587+0000 7fa523c45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:58:30.600 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.600+0000 7f4054643040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:58:30.641 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.642+0000 7fa523c45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:58:30.654 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.654+0000 7f4054643040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:58:30.707 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.707+0000 7fa523c45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:58:30.720 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.720+0000 7f4054643040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:58:30.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.750+0000 7fd8e0a00640 1 Processor -- start 2024-03-28T17:58:30.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.750+0000 7fd8e0a00640 1 -- start start 2024-03-28T17:58:30.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8e0a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8e0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8dc0d0990 0x7fd8dc0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8e0a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0d2630 con 0x7fd8dc0d36d0 2024-03-28T17:58:30.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8e0a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0d2400 con 0x7fd8dc0d0990 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8e0a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0d3240 con 0x7fd8dc15df40 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8d1e00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd8dc0d36d0 0x7fd8dc0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:57124/0 (socket says 172.21.15.81:57124) 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8d1e00640 1 -- 172.21.15.81:0/1096700012 learned_addr learned my addr 172.21.15.81:0/1096700012 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8da800640 1 --2- 172.21.15.81:0/1096700012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.752+0000 7fd8db200640 1 --2- 172.21.15.81:0/1096700012 >> v2:172.21.15.62:3300/0 conn(0x7fd8dc0d0990 0x7fd8dc0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8d9e00640 1 -- 172.21.15.81:0/1096700012 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151715562 0 0) 0x7fd8dc0d2630 con 0x7fd8dc0d36d0 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8d9e00640 1 -- 172.21.15.81:0/1096700012 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8c0003520 con 0x7fd8dc0d36d0 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8da800640 1 -- 172.21.15.81:0/1096700012 >> v2:172.21.15.62:3300/0 conn(0x7fd8dc0d0990 msgr2=0x7fd8dc0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8da800640 1 --2- 172.21.15.81:0/1096700012 >> v2:172.21.15.62:3300/0 conn(0x7fd8dc0d0990 0x7fd8dc0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8da800640 1 -- 172.21.15.81:0/1096700012 >> v1:172.21.15.81:6789/0 conn(0x7fd8dc0d36d0 legacy=0x7fd8dc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.753+0000 7fd8da800640 1 -- 172.21.15.81:0/1096700012 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0d2910 con 0x7fd8dc15df40 2024-03-28T17:58:30.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.754+0000 7fd8da800640 1 --2- 172.21.15.81:0/1096700012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0b9730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009d90 tx=0x7fd8cc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76eb629f5383e2d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.754+0000 7fd8d9e00640 1 -- 172.21.15.81:0/1096700012 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc039070 con 0x7fd8dc15df40 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.754+0000 7fd8d9e00640 1 -- 172.21.15.81:0/1096700012 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc02cc50 con 0x7fd8dc15df40 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.754+0000 7fd8d9e00640 1 -- 172.21.15.81:0/1096700012 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc034440 con 0x7fd8dc15df40 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 -- 172.21.15.81:0/1096700012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 msgr2=0x7fd8dc0b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/1096700012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0b9730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009d90 tx=0x7fd8cc02bea0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 -- 172.21.15.81:0/1096700012 shutdown_connections 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/1096700012 >> v2:172.21.15.62:3300/0 conn(0x7fd8dc0d0990 0x7fd8dc0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/1096700012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0b9730 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 -- 172.21.15.81:0/1096700012 >> 172.21.15.81:0/1096700012 conn(0x7fd8dc071cf0 msgr2=0x7fd8dc066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 -- 172.21.15.81:0/1096700012 shutdown_connections 2024-03-28T17:58:30.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.755+0000 7fd8e0a00640 1 -- 172.21.15.81:0/1096700012 wait complete. 2024-03-28T17:58:30.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.757+0000 7fd8e0a00640 1 Processor -- start 2024-03-28T17:58:30.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.758+0000 7fd8e0a00640 1 -- start start 2024-03-28T17:58:30.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8dc0d0990 0x7fd8dc157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8dc0d36d0 0x7fd8dc158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:30.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0b6cd0 con 0x7fd8dc0d0990 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0d2ed0 con 0x7fd8dc0d36d0 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8e0a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0b6e50 con 0x7fd8dc15df40 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8db200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8dc0d36d0 0x7fd8dc158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.759+0000 7fd8db200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:50722/0 (socket says 172.21.15.81:50722) 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 -- 172.21.15.81:0/2877804831 learned_addr learned my addr 172.21.15.81:0/2877804831 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8da800640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8dc0d0990 0x7fd8dc157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 -- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8dc0d36d0 msgr2=0x7fd8dc158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8dc0d36d0 0x7fd8dc158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 -- 172.21.15.81:0/2877804831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8dc0d0990 msgr2=0x7fd8dc157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8dc0d0990 0x7fd8dc157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 -- 172.21.15.81:0/2877804831 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc157df0 con 0x7fd8dc15df40 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8db200640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400d5d0 tx=0x7fd8d400da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40125f0 con 0x7fd8dc15df40 2024-03-28T17:58:30.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4019070 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc1580d0 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.760+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40129d0 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.761+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc158690 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.761+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd8d4012b70 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.761+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd8d401e880 con 0x7fd8dc15df40 2024-03-28T17:58:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.762+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8dc04f400 con 0x7fd8dc15df40 2024-03-28T17:58:30.763 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.764+0000 7fa523c45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:58:30.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.766+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd8d401eb60 con 0x7fd8dc15df40 2024-03-28T17:58:30.772 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.772+0000 7f4054643040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:58:30.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.854+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "584b7a16-2729-4cf0-a52a-2384928197c9", "id": 2} v 0) v1 -- 0x7fd8dc15bce0 con 0x7fd8dc15df40 2024-03-28T17:58:30.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.860+0000 7fd8d3400640 1 -- 172.21.15.81:0/2877804831 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "584b7a16-2729-4cf0-a52a-2384928197c9", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd8d401e460 con 0x7fd8dc15df40 2024-03-28T17:58:30.859 INFO:teuthology.orchestra.run.smithi081.stdout:2 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 msgr2=0x7fd8dc0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400d5d0 tx=0x7fd8d400da90 comp rx=0 tx=0).stop 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 shutdown_connections 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8dc15df40 0x7fd8dc0c9dc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8dc0d36d0 0x7fd8dc158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 --2- 172.21.15.81:0/2877804831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8dc0d0990 0x7fd8dc157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 >> 172.21.15.81:0/2877804831 conn(0x7fd8dc071cf0 msgr2=0x7fd8dc0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 shutdown_connections 2024-03-28T17:58:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:30.861+0000 7fd8e0a00640 1 -- 172.21.15.81:0/2877804831 wait complete. 2024-03-28T17:58:30.875 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new a08a6c7d-b71a-4329-8849-5add820b36ba 3 2024-03-28T17:58:30.906 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.907+0000 7fa523c45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:58:30.910 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.910+0000 7f4054643040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:58:30.991 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:30.991+0000 7fa523c45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:58:30.994 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:30.994+0000 7f4054643040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:58:31.048 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.049+0000 7fa523c45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:58:31.051 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.051+0000 7f4054643040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:58:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.101+0000 7f16ca000640 1 Processor -- start 2024-03-28T17:58:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.102+0000 7f16ca000640 1 -- start start 2024-03-28T17:58:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16ca000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f16c415e0f0 0x7f16c40d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16ca000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16c40d2350 0x7f16c40d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16ca000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40d0eb0 con 0x7f16c40d0a20 2024-03-28T17:58:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16ca000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c40d33e0 con 0x7f16c40d2350 2024-03-28T17:58:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16ca000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c40d27d0 con 0x7f16c415e0f0 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16c8c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f16c415e0f0 0x7f16c40d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16c8c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f16c415e0f0 0x7f16c40d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45638/0 (socket says 172.21.15.81:45638) 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16c8c00640 1 -- 172.21.15.81:0/3812519473 learned_addr learned my addr 172.21.15.81:0/3812519473 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.103+0000 7f16c3e00640 1 --2- 172.21.15.81:0/3812519473 >> v2:172.21.15.62:3300/0 conn(0x7f16c40d2350 0x7f16c40d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.104+0000 7f16c8c00640 1 -- 172.21.15.81:0/3812519473 >> v2:172.21.15.62:3300/0 conn(0x7f16c40d2350 msgr2=0x7f16c40d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.104+0000 7f16c8c00640 1 --2- 172.21.15.81:0/3812519473 >> v2:172.21.15.62:3300/0 conn(0x7f16c40d2350 0x7f16c40d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.104+0000 7f16c8c00640 1 -- 172.21.15.81:0/3812519473 >> v1:172.21.15.62:6789/0 conn(0x7f16c40d0a20 legacy=0x7f16c40b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.104+0000 7f16c8c00640 1 -- 172.21.15.81:0/3812519473 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40d2ab0 con 0x7f16c415e0f0 2024-03-28T17:58:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.105+0000 7f16c8c00640 1 --2- 172.21.15.81:0/3812519473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c415e0f0 0x7f16c40d1e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d3b10 tx=0x7f16b802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95639b9327a6e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.105+0000 7f16c3400640 1 -- 172.21.15.81:0/3812519473 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8034070 con 0x7f16c415e0f0 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.105+0000 7f16c3400640 1 -- 172.21.15.81:0/3812519473 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8030860 con 0x7f16c415e0f0 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.105+0000 7f16c3400640 1 -- 172.21.15.81:0/3812519473 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8030c30 con 0x7f16c415e0f0 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 -- 172.21.15.81:0/3812519473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c415e0f0 msgr2=0x7f16c40d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 --2- 172.21.15.81:0/3812519473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c415e0f0 0x7f16c40d1e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d3b10 tx=0x7f16b802b9a0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 -- 172.21.15.81:0/3812519473 shutdown_connections 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 --2- 172.21.15.81:0/3812519473 >> v2:172.21.15.62:3300/0 conn(0x7f16c40d2350 0x7f16c40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 --2- 172.21.15.81:0/3812519473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c415e0f0 0x7f16c40d1e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 -- 172.21.15.81:0/3812519473 >> 172.21.15.81:0/3812519473 conn(0x7f16c4071cf0 msgr2=0x7f16c4066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 -- 172.21.15.81:0/3812519473 shutdown_connections 2024-03-28T17:58:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.106+0000 7f16ca000640 1 -- 172.21.15.81:0/3812519473 wait complete. 2024-03-28T17:58:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.107+0000 7f16ca000640 1 Processor -- start 2024-03-28T17:58:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 -- start start 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c40d0a20 0x7f16c414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16c415e0f0 0x7f16c40aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c415e4d0 con 0x7f16c40d0a20 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40d3070 con 0x7f16c40d2350 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.108+0000 7f16ca000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40ce650 con 0x7f16c415e0f0 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c9600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16c415e0f0 0x7f16c40aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45648/0 (socket says 172.21.15.81:45648) 2024-03-28T17:58:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 -- 172.21.15.81:0/3706901893 learned_addr learned my addr 172.21.15.81:0/3706901893 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c8c00640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c40d0a20 0x7f16c414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 -- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16c415e0f0 msgr2=0x7f16c40aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16c415e0f0 0x7f16c40aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 -- 172.21.15.81:0/3706901893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c40d0a20 msgr2=0x7f16c414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c40d0a20 0x7f16c414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 -- 172.21.15.81:0/3706901893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c414d7d0 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.109+0000 7f16c3e00640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d800 tx=0x7f16b000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000b6c0 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b0019070 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000ba80 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c4160720 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4160fc0 con 0x7f16c40d2350 2024-03-28T17:58:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f16b000bc20 con 0x7f16c40d2350 2024-03-28T17:58:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.110+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f16b001e8f0 con 0x7f16c40d2350 2024-03-28T17:58:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.111+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4160b10 con 0x7f16c40d2350 2024-03-28T17:58:31.115 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.115+0000 7f4054643040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:58:31.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.116+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f16b001eaf0 con 0x7f16c40d2350 2024-03-28T17:58:31.116 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.116+0000 7fa523c45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:58:31.173 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.174+0000 7fa523c45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:58:31.174 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.173+0000 7f4054643040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:58:31.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.195+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a08a6c7d-b71a-4329-8849-5add820b36ba", "id": 3} v 0) v1 -- 0x7f16c40b5b10 con 0x7f16c40d2350 2024-03-28T17:58:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.202+0000 7f16c0c00640 1 -- 172.21.15.81:0/3706901893 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a08a6c7d-b71a-4329-8849-5add820b36ba", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f16b001e460 con 0x7f16c40d2350 2024-03-28T17:58:31.202 INFO:teuthology.orchestra.run.smithi081.stdout:3 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 msgr2=0x7f16c40aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d800 tx=0x7f16b000dcc0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 shutdown_connections 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f16c415e0f0 0x7f16c40aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16c40d2350 0x7f16c40aa5d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 --2- 172.21.15.81:0/3706901893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c40d0a20 0x7f16c414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 >> 172.21.15.81:0/3706901893 conn(0x7f16c4071cf0 msgr2=0x7f16c40b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 shutdown_connections 2024-03-28T17:58:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.204+0000 7f16ca000640 1 -- 172.21.15.81:0/3706901893 wait complete. 2024-03-28T17:58:31.220 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 9a37caf2-9668-48c1-a579-29fa3626208f 4 2024-03-28T17:58:31.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.400+0000 7f264f200640 1 Processor -- start 2024-03-28T17:58:31.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.401+0000 7f264f200640 1 -- start start 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264f200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f265015df40 0x7f26500b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26500d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264f200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26500d2630 con 0x7f26500d36d0 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264f200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26500d2400 con 0x7f265015df40 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d3240 con 0x7f26500d0990 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264de00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f265015df40 0x7f26500b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264e800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26500d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264de00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f265015df40 0x7f26500b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45656/0 (socket says 172.21.15.81:45656) 2024-03-28T17:58:31.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.402+0000 7f264de00640 1 -- 172.21.15.81:0/1021241494 learned_addr learned my addr 172.21.15.81:0/1021241494 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264d400640 1 -- 172.21.15.81:0/1021241494 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405810386 0 0) 0x7f26500d2630 con 0x7f26500d36d0 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264de00640 1 -- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 msgr2=0x7f26500d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264de00640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26500d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264de00640 1 -- 172.21.15.81:0/1021241494 >> v1:172.21.15.81:6789/0 conn(0x7f26500d36d0 legacy=0x7f26500d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264de00640 1 -- 172.21.15.81:0/1021241494 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500d2910 con 0x7f265015df40 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.403+0000 7f264e800640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26500d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.404+0000 7f264de00640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b9730 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26500d33c0 tx=0x7f264002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac65f814836ef174 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.404+0000 7f264d400640 1 -- 172.21.15.81:0/1021241494 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640034070 con 0x7f265015df40 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.404+0000 7f264d400640 1 -- 172.21.15.81:0/1021241494 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2640030840 con 0x7f265015df40 2024-03-28T17:58:31.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.404+0000 7f264d400640 1 -- 172.21.15.81:0/1021241494 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640030c10 con 0x7f265015df40 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 -- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 msgr2=0x7f26500b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b9730 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26500d33c0 tx=0x7f264002b980 comp rx=0 tx=0).stop 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 -- 172.21.15.81:0/1021241494 shutdown_connections 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26500d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 --2- 172.21.15.81:0/1021241494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b9730 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 -- 172.21.15.81:0/1021241494 >> 172.21.15.81:0/1021241494 conn(0x7f2650071cf0 msgr2=0x7f2650066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 -- 172.21.15.81:0/1021241494 shutdown_connections 2024-03-28T17:58:31.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.405+0000 7f264f200640 1 -- 172.21.15.81:0/1021241494 wait complete. 2024-03-28T17:58:31.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.408+0000 7f264f200640 1 Processor -- start 2024-03-28T17:58:31.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.408+0000 7f264f200640 1 -- start start 2024-03-28T17:58:31.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 0x7f26500b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500b6cd0 con 0x7f26500d36d0 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d2ed0 con 0x7f265015df40 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26500b6e50 con 0x7f26500d0990 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:58908/0 (socket says 172.21.15.81:58908) 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264de00640 1 -- 172.21.15.81:0/2818269912 learned_addr learned my addr 172.21.15.81:0/2818269912 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.409+0000 7f264e800640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f2647e00640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 0x7f26500b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 -- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 msgr2=0x7f26500b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 -- 172.21.15.81:0/2818269912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 msgr2=0x7f26500b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 0x7f26500b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 -- 172.21.15.81:0/2818269912 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26501535e0 con 0x7f26500d0990 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264e800640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f2647e00640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 0x7f26500b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f264de00640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2640000c00 tx=0x7f26400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.410+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640034040 con 0x7f26500d0990 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264003ab90 con 0x7f26500d0990 2024-03-28T17:58:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640004030 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26501552e0 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2650154a40 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f264004b060 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.411+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2640003d50 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.412+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265004f400 con 0x7f26500d0990 2024-03-28T17:58:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.414+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2640063dd0 con 0x7f26500d0990 2024-03-28T17:58:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.493+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9a37caf2-9668-48c1-a579-29fa3626208f", "id": 4} v 0) v1 -- 0x7f26500c9040 con 0x7f26500d0990 2024-03-28T17:58:31.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.500+0000 7f2646000640 1 -- 172.21.15.81:0/2818269912 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a37caf2-9668-48c1-a579-29fa3626208f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2640052030 con 0x7f26500d0990 2024-03-28T17:58:31.501 INFO:teuthology.orchestra.run.smithi081.stdout:4 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 msgr2=0x7f26501530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2640000c00 tx=0x7f26400046c0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 shutdown_connections 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f265015df40 0x7f26500b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26500d36d0 0x7f26500b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 --2- 172.21.15.81:0/2818269912 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26500d0990 0x7f26501530a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 >> 172.21.15.81:0/2818269912 conn(0x7f2650071cf0 msgr2=0x7f26500c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 shutdown_connections 2024-03-28T17:58:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.502+0000 7f264f200640 1 -- 172.21.15.81:0/2818269912 wait complete. 2024-03-28T17:58:31.518 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 78fef44b-4df0-406d-8b8f-026bc08bc74c 5 2024-03-28T17:58:31.646 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.646+0000 7f4054643040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:58:31.650 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.650+0000 7fa523c45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:58:31.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.705+0000 7fdd0da00640 1 Processor -- start 2024-03-28T17:58:31.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.705+0000 7fdd0da00640 1 -- start start 2024-03-28T17:58:31.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.706+0000 7fdd0da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd080d04e0 0x7fdd080d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.706+0000 7fdd0da00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdd080b7190 0x7fdd080b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.706+0000 7fdd0da00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd080d2520 con 0x7fdd0815e0f0 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.706+0000 7fdd0da00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080d33e0 con 0x7fdd080d04e0 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.706+0000 7fdd0da00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080d27d0 con 0x7fdd080b7190 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd07e00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdd0815e0f0 0x7fdd080d1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34300/0 (socket says 172.21.15.81:34300) 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd07e00640 1 -- 172.21.15.81:0/2197098945 learned_addr learned my addr 172.21.15.81:0/2197098945 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd0d000640 1 --2- 172.21.15.81:0/2197098945 >> v2:172.21.15.81:3300/0 conn(0x7fdd080b7190 0x7fdd080b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd07400640 1 --2- 172.21.15.81:0/2197098945 >> v2:172.21.15.62:3300/0 conn(0x7fdd080d04e0 0x7fdd080d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd0d000640 1 -- 172.21.15.81:0/2197098945 >> v2:172.21.15.62:3300/0 conn(0x7fdd080d04e0 msgr2=0x7fdd080d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd0d000640 1 --2- 172.21.15.81:0/2197098945 >> v2:172.21.15.62:3300/0 conn(0x7fdd080d04e0 0x7fdd080d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd0d000640 1 -- 172.21.15.81:0/2197098945 >> v1:172.21.15.81:6789/0 conn(0x7fdd0815e0f0 legacy=0x7fdd080d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.707+0000 7fdd0d000640 1 -- 172.21.15.81:0/2197098945 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080d2ab0 con 0x7fdd080b7190 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.708+0000 7fdd0d000640 1 --2- 172.21.15.81:0/2197098945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080b7190 0x7fdd080b7620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42cef74ab85fe629 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd06a00640 1 -- 172.21.15.81:0/2197098945 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00039070 con 0x7fdd080b7190 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd06a00640 1 -- 172.21.15.81:0/2197098945 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0002cc60 con 0x7fdd080b7190 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd06a00640 1 -- 172.21.15.81:0/2197098945 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00034660 con 0x7fdd080b7190 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 -- 172.21.15.81:0/2197098945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080b7190 msgr2=0x7fdd080b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 --2- 172.21.15.81:0/2197098945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080b7190 0x7fdd080b7620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002beb0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 -- 172.21.15.81:0/2197098945 shutdown_connections 2024-03-28T17:58:31.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 --2- 172.21.15.81:0/2197098945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080b7190 0x7fdd080b7620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 --2- 172.21.15.81:0/2197098945 >> v2:172.21.15.62:3300/0 conn(0x7fdd080d04e0 0x7fdd080d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 -- 172.21.15.81:0/2197098945 >> 172.21.15.81:0/2197098945 conn(0x7fdd08071cf0 msgr2=0x7fdd08066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.709+0000 7fdd0da00640 1 -- 172.21.15.81:0/2197098945 shutdown_connections 2024-03-28T17:58:31.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.710+0000 7fdd0da00640 1 -- 172.21.15.81:0/2197098945 wait complete. 2024-03-28T17:58:31.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.711+0000 7fdd0da00640 1 Processor -- start 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.711+0000 7fdd0da00640 1 -- start start 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 0x7fdd080c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 0x7fdd080cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080d0e60 con 0x7fdd080b7190 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080d3070 con 0x7fdd080d04e0 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0da00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0815e4d0 con 0x7fdd0815e0f0 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd07400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd0d000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 0x7fdd080cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd07400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45670/0 (socket says 172.21.15.81:45670) 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd07400640 1 -- 172.21.15.81:0/3844210915 learned_addr learned my addr 172.21.15.81:0/3844210915 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.712+0000 7fdd07e00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 0x7fdd080c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 -- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 msgr2=0x7fdd080cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 0x7fdd080cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 -- 172.21.15.81:0/3844210915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 msgr2=0x7fdd080c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 0x7fdd080c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 -- 172.21.15.81:0/3844210915 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080cbe50 con 0x7fdd080d04e0 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07e00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 0x7fdd080c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd0d000640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 0x7fdd080cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd07400640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400d800 tx=0x7fdcf400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:31.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400b6c0 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4019070 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd080cc130 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.713+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400ba80 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.714+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd080cc6f0 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.714+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdcf400bc20 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.714+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fdcf401e9e0 con 0x7fdd080d04e0 2024-03-28T17:58:31.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.715+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0804f400 con 0x7fdd080d04e0 2024-03-28T17:58:31.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.717+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdcf401ebe0 con 0x7fdd080d04e0 2024-03-28T17:58:31.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.796+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd new", "uuid": "78fef44b-4df0-406d-8b8f-026bc08bc74c", "id": 5} v 0) v1 -- 0x7fdd0815d6c0 con 0x7fdd080d04e0 2024-03-28T17:58:31.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.803+0000 7fdcffe00640 1 -- 172.21.15.81:0/3844210915 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "78fef44b-4df0-406d-8b8f-026bc08bc74c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdcf401e460 con 0x7fdd080d04e0 2024-03-28T17:58:31.803 INFO:teuthology.orchestra.run.smithi081.stdout:5 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 msgr2=0x7fdd080cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400d800 tx=0x7fdcf400dcc0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 shutdown_connections 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdd0815e0f0 0x7fdd080cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd080d04e0 0x7fdd080cd400 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 --2- 172.21.15.81:0/3844210915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd080b7190 0x7fdd080c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:31.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 >> 172.21.15.81:0/3844210915 conn(0x7fdd08071cf0 msgr2=0x7fdd080ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:31.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 shutdown_connections 2024-03-28T17:58:31.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:31.805+0000 7fdd0da00640 1 -- 172.21.15.81:0/3844210915 wait complete. 2024-03-28T17:58:31.815 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.815+0000 7f4054643040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:58:31.819 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.819+0000 7fa523c45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:58:31.821 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new a9a414ab-f06e-4bbb-8f8b-2939857fca35 6 2024-03-28T17:58:31.957 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:31.957+0000 7fa523c45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:58:31.968 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:31.968+0000 7f4054643040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:58:32.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.008+0000 7f4d7ee00640 1 Processor -- start 2024-03-28T17:58:32.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.008+0000 7f4d7ee00640 1 -- start start 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7ee00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4d7815e0f0 0x7f4d780d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7ee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d780d0bf0 0x7f4d780b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7ee00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d780d27d0 con 0x7f4d780d2350 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7ee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d780d33e0 con 0x7f4d780d0bf0 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7ee00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d780d2ab0 con 0x7f4d7815e0f0 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7da00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4d7815e0f0 0x7f4d780d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7da00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4d7815e0f0 0x7f4d780d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45672/0 (socket says 172.21.15.81:45672) 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.010+0000 7f4d7da00640 1 -- 172.21.15.81:0/2769547473 learned_addr learned my addr 172.21.15.81:0/2769547473 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.011+0000 7f4d7e400640 1 --2- 172.21.15.81:0/2769547473 >> v2:172.21.15.62:3300/0 conn(0x7f4d780d0bf0 0x7f4d780b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.011+0000 7f4d7da00640 1 -- 172.21.15.81:0/2769547473 >> v2:172.21.15.62:3300/0 conn(0x7f4d780d0bf0 msgr2=0x7f4d780b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:32.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.011+0000 7f4d7da00640 1 --2- 172.21.15.81:0/2769547473 >> v2:172.21.15.62:3300/0 conn(0x7f4d780d0bf0 0x7f4d780b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.011+0000 7f4d7da00640 1 -- 172.21.15.81:0/2769547473 >> v1:172.21.15.81:6789/0 conn(0x7f4d780d2350 legacy=0x7f4d780d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.011+0000 7f4d7da00640 1 -- 172.21.15.81:0/2769547473 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d780d2d90 con 0x7f4d7815e0f0 2024-03-28T17:58:32.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.012+0000 7f4d7da00640 1 --2- 172.21.15.81:0/2769547473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d7815e0f0 0x7f4d780d1e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4d780d3b10 tx=0x7f4d6c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cce4b9bc475de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.012+0000 7f4d77e00640 1 -- 172.21.15.81:0/2769547473 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c034070 con 0x7f4d7815e0f0 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.012+0000 7f4d77e00640 1 -- 172.21.15.81:0/2769547473 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c030860 con 0x7f4d7815e0f0 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.012+0000 7f4d77e00640 1 -- 172.21.15.81:0/2769547473 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c030c30 con 0x7f4d7815e0f0 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 -- 172.21.15.81:0/2769547473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d7815e0f0 msgr2=0x7f4d780d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/2769547473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d7815e0f0 0x7f4d780d1e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4d780d3b10 tx=0x7f4d6c02b9a0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 -- 172.21.15.81:0/2769547473 shutdown_connections 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/2769547473 >> v2:172.21.15.62:3300/0 conn(0x7f4d780d0bf0 0x7f4d780b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/2769547473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d7815e0f0 0x7f4d780d1e10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 -- 172.21.15.81:0/2769547473 >> 172.21.15.81:0/2769547473 conn(0x7f4d78071cf0 msgr2=0x7f4d78066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 -- 172.21.15.81:0/2769547473 shutdown_connections 2024-03-28T17:58:32.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.013+0000 7f4d7ee00640 1 -- 172.21.15.81:0/2769547473 wait complete. 2024-03-28T17:58:32.015 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.014+0000 7fa523c45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.015+0000 7f4d7ee00640 1 Processor -- start 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.015+0000 7f4d7ee00640 1 -- start start 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 0x7f4d7815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 0x7f4d780cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780ce650 con 0x7f4d7815e0f0 2024-03-28T17:58:32.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7815e4d0 con 0x7f4d780d2350 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7ee00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780ce7d0 con 0x7f4d780d0bf0 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7da00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 0x7f4d7815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7da00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:58924/0 (socket says 172.21.15.81:58924) 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7da00640 1 -- 172.21.15.81:0/867130549 learned_addr learned my addr 172.21.15.81:0/867130549 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7e400640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 0x7f4d780cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 -- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 msgr2=0x7f4d7815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 -- 172.21.15.81:0/867130549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 msgr2=0x7f4d780cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 0x7f4d780cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7d000640 1 -- 172.21.15.81:0/867130549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7815f630 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7e400640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 0x7f4d780cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.016+0000 7f4d7da00640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d7d000640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 0x7f4d7815cb50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c340 tx=0x7f4d6400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640052d0 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d64005490 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78160590 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64007260 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d78160b50 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d64007400 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.017+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4d640067d0 con 0x7f4d780d2350 2024-03-28T17:58:32.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.018+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7804f400 con 0x7f4d780d2350 2024-03-28T17:58:32.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.021+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d640035f0 con 0x7f4d780d2350 2024-03-28T17:58:32.025 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.025+0000 7f4054643040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:58:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.100+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9a414ab-f06e-4bbb-8f8b-2939857fca35", "id": 6} v 0) v1 -- 0x7f4d780b8e70 con 0x7f4d780d2350 2024-03-28T17:58:32.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.106+0000 7f4d75600640 1 -- 172.21.15.81:0/867130549 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9a414ab-f06e-4bbb-8f8b-2939857fca35", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4d64006aa0 con 0x7f4d780d2350 2024-03-28T17:58:32.106 INFO:teuthology.orchestra.run.smithi081.stdout:6 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 msgr2=0x7f4d7815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 0x7f4d7815cb50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c340 tx=0x7f4d6400c800 comp rx=0 tx=0).stop 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 shutdown_connections 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d7815e0f0 0x7f4d780cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d780d2350 0x7f4d7815cb50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 --2- 172.21.15.81:0/867130549 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4d780d0bf0 0x7f4d7815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 >> 172.21.15.81:0/867130549 conn(0x7f4d78071cf0 msgr2=0x7f4d780c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 shutdown_connections 2024-03-28T17:58:32.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.107+0000 7f4d7ee00640 1 -- 172.21.15.81:0/867130549 wait complete. 2024-03-28T17:58:32.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 67867055-2853-4975-895d-2f444fc83ca1 7 2024-03-28T17:58:32.150 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.151+0000 7fa523c45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:58:32.159 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.159+0000 7f4054643040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:58:32.268 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.268+0000 7fa523c45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:58:32.275 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.275+0000 7f4054643040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.306+0000 7f6d8f000640 1 Processor -- start 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.306+0000 7f6d8f000640 1 -- start start 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d8816e140 0x7f6d880b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 0x7f6d8816e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b4360 con 0x7f6d8816e140 2024-03-28T17:58:32.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b32c0 con 0x7f6d8808b830 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.307+0000 7f6d8f000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b35a0 con 0x7f6d880727d0 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.308+0000 7f6d8d000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 0x7f6d8816e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.308+0000 7f6d87400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.308+0000 7f6d87400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.81:58934/0 (socket says 172.21.15.81:58934) 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.308+0000 7f6d8d000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 0x7f6d8816e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45698/0 (socket says 172.21.15.81:45698) 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.308+0000 7f6d87e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d8816e140 0x7f6d880b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.317+0000 7f6d87400640 1 -- 172.21.15.81:0/4207651204 learned_addr learned my addr 172.21.15.81:0/4207651204 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.318+0000 7f6d87400640 1 -- 172.21.15.81:0/4207651204 >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 msgr2=0x7f6d8816e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.318+0000 7f6d87400640 1 --2- 172.21.15.81:0/4207651204 >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 0x7f6d8816e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.318+0000 7f6d87400640 1 -- 172.21.15.81:0/4207651204 >> v2:172.21.15.62:3300/0 conn(0x7f6d8816e140 msgr2=0x7f6d880b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:32.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.318+0000 7f6d87400640 1 --2- 172.21.15.81:0/4207651204 >> v2:172.21.15.62:3300/0 conn(0x7f6d8816e140 0x7f6d880b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.318+0000 7f6d87400640 1 -- 172.21.15.81:0/4207651204 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880b3880 con 0x7f6d880727d0 2024-03-28T17:58:32.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.319+0000 7f6d87400640 1 --2- 172.21.15.81:0/4207651204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b4da0 tx=0x7f6d7402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fdb14bad0f7c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.319+0000 7f6d86a00640 1 -- 172.21.15.81:0/4207651204 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74039070 con 0x7f6d880727d0 2024-03-28T17:58:32.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.319+0000 7f6d86a00640 1 -- 172.21.15.81:0/4207651204 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7402cc40 con 0x7f6d880727d0 2024-03-28T17:58:32.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.319+0000 7f6d86a00640 1 -- 172.21.15.81:0/4207651204 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74034440 con 0x7f6d880727d0 2024-03-28T17:58:32.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 -- 172.21.15.81:0/4207651204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 msgr2=0x7f6d8808b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 --2- 172.21.15.81:0/4207651204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b4da0 tx=0x7f6d7402be90 comp rx=0 tx=0).stop 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 -- 172.21.15.81:0/4207651204 shutdown_connections 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 --2- 172.21.15.81:0/4207651204 >> v2:172.21.15.81:3300/0 conn(0x7f6d8808b830 0x7f6d8816e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 --2- 172.21.15.81:0/4207651204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d880727d0 0x7f6d8808b2f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 --2- 172.21.15.81:0/4207651204 >> v2:172.21.15.62:3300/0 conn(0x7f6d8816e140 0x7f6d880b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 -- 172.21.15.81:0/4207651204 >> 172.21.15.81:0/4207651204 conn(0x7f6d880b1650 msgr2=0x7f6d880b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 -- 172.21.15.81:0/4207651204 shutdown_connections 2024-03-28T17:58:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.320+0000 7f6d8f000640 1 -- 172.21.15.81:0/4207651204 wait complete. 2024-03-28T17:58:32.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.322+0000 7f6d8f000640 1 Processor -- start 2024-03-28T17:58:32.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.322+0000 7f6d8f000640 1 -- start start 2024-03-28T17:58:32.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8808b830 0x7f6d8815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d8816e140 0x7f6d8816c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8816e520 con 0x7f6d8808b830 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b3e40 con 0x7f6d880727d0 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8f000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88072c10 con 0x7f6d8816e140 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d87e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d8d000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d8816e140 0x7f6d8816c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d87e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45700/0 (socket says 172.21.15.81:45700) 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.323+0000 7f6d87e00640 1 -- 172.21.15.81:0/3040672609 learned_addr learned my addr 172.21.15.81:0/3040672609 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-28T17:58:32.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87400640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8808b830 0x7f6d8815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 -- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d8816e140 msgr2=0x7f6d8816c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d8816e140 0x7f6d8816c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 -- 172.21.15.81:0/3040672609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8808b830 msgr2=0x7f6d8815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8808b830 0x7f6d8815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 -- 172.21.15.81:0/3040672609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880b1150 con 0x7f6d880727d0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d87e00640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d780097c0 tx=0x7f6d78009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78018020 con 0x7f6d880727d0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800ebb0 con 0x7f6d880727d0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78013440 con 0x7f6d880727d0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.324+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d88163d50 con 0x7f6d880727d0 2024-03-28T17:58:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.325+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88164310 con 0x7f6d880727d0 2024-03-28T17:58:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.325+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6d7801b020 con 0x7f6d880727d0 2024-03-28T17:58:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.325+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f6d7801d7c0 con 0x7f6d880727d0 2024-03-28T17:58:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.326+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d880680c0 con 0x7f6d880727d0 2024-03-28T17:58:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.328+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6d78022020 con 0x7f6d880727d0 2024-03-28T17:58:32.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.407+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd new", "uuid": "67867055-2853-4975-895d-2f444fc83ca1", "id": 7} v 0) v1 -- 0x7f6d880cf050 con 0x7f6d880727d0 2024-03-28T17:58:32.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.413+0000 7f6d7fe00640 1 -- 172.21.15.81:0/3040672609 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "67867055-2853-4975-895d-2f444fc83ca1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d7801daa0 con 0x7f6d880727d0 2024-03-28T17:58:32.413 INFO:teuthology.orchestra.run.smithi081.stdout:7 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 msgr2=0x7f6d88163650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d780097c0 tx=0x7f6d78009c80 comp rx=0 tx=0).stop 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 shutdown_connections 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6d8816e140 0x7f6d8816c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8808b830 0x7f6d8815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 --2- 172.21.15.81:0/3040672609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d880727d0 0x7f6d88163650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 >> 172.21.15.81:0/3040672609 conn(0x7f6d880b1650 msgr2=0x7f6d880cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 shutdown_connections 2024-03-28T17:58:32.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:58:32.415+0000 7f6d8f000640 1 -- 172.21.15.81:0/3040672609 wait complete. 2024-03-28T17:58:32.431 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-28T17:58:32.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-28T17:58:32.439 INFO:tasks.ceph.osd.0:Started 2024-03-28T17:58:32.439 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-28T17:58:32.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-28T17:58:32.447 INFO:tasks.ceph.osd.1:Started 2024-03-28T17:58:32.447 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-28T17:58:32.447 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-28T17:58:32.452 INFO:tasks.ceph.osd.2:Started 2024-03-28T17:58:32.452 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-28T17:58:32.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-28T17:58:32.457 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.456+0000 7fa523c45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:58:32.457 INFO:tasks.ceph.osd.3:Started 2024-03-28T17:58:32.458 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-28T17:58:32.458 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-28T17:58:32.465 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.464+0000 7f4054643040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:58:32.465 INFO:tasks.ceph.osd.4:Started 2024-03-28T17:58:32.465 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-28T17:58:32.466 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-28T17:58:32.472 INFO:tasks.ceph.osd.5:Started 2024-03-28T17:58:32.473 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-28T17:58:32.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-28T17:58:32.478 INFO:tasks.ceph.osd.6:Started 2024-03-28T17:58:32.478 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-28T17:58:32.478 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-28T17:58:32.484 INFO:tasks.ceph.osd.7:Started 2024-03-28T17:58:32.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:32.517 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.518+0000 7fa523c45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:58:32.541 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.541+0000 7f4054643040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:58:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.709+0000 7f654f600640 1 Processor -- start 2024-03-28T17:58:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.709+0000 7f654f600640 1 -- start start 2024-03-28T17:58:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f654f600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65480d0980 0x7f65480cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f654f600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480cd8e0 con 0x7f65480ceb00 2024-03-28T17:58:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f654f600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480cdbc0 con 0x7f65480d05a0 2024-03-28T17:58:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f654f600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480cdea0 con 0x7f65480d0980 2024-03-28T17:58:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f6547e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65480d0980 0x7f65480cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f6547e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65480d0980 0x7f65480cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52768/0 (socket says 172.21.15.62:52768) 2024-03-28T17:58:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.710+0000 7f6547e00640 1 -- 172.21.15.62:0/2044519131 learned_addr learned my addr 172.21.15.62:0/2044519131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.711+0000 7f6547e00640 1 -- 172.21.15.62:0/2044519131 >> v1:172.21.15.81:6789/0 conn(0x7f65480d05a0 legacy=0x7f65480ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.711+0000 7f6547e00640 1 -- 172.21.15.62:0/2044519131 >> v1:172.21.15.62:6789/0 conn(0x7f65480ceb00 legacy=0x7f654806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.711+0000 7f6547e00640 1 -- 172.21.15.62:0/2044519131 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654806b1d0 con 0x7f65480d0980 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.712+0000 7f6547e00640 1 --2- 172.21.15.62:0/2044519131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65480cfb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6534009870 tx=0x7f653402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4367dedfa34fd6f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.712+0000 7f6547400640 1 -- 172.21.15.62:0/2044519131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534034070 con 0x7f65480d0980 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.712+0000 7f6547400640 1 -- 172.21.15.62:0/2044519131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6534030860 con 0x7f65480d0980 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.712+0000 7f6547400640 1 -- 172.21.15.62:0/2044519131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534030c30 con 0x7f65480d0980 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 -- 172.21.15.62:0/2044519131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 msgr2=0x7f65480cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 --2- 172.21.15.62:0/2044519131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65480cfb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6534009870 tx=0x7f653402b9a0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 -- 172.21.15.62:0/2044519131 shutdown_connections 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 --2- 172.21.15.62:0/2044519131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65480cfb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 -- 172.21.15.62:0/2044519131 >> 172.21.15.62:0/2044519131 conn(0x7f65480b38e0 msgr2=0x7f65480b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 -- 172.21.15.62:0/2044519131 shutdown_connections 2024-03-28T17:58:32.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.713+0000 7f654f600640 1 -- 172.21.15.62:0/2044519131 wait complete. 2024-03-28T17:58:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.714+0000 7f654f600640 1 Processor -- start 2024-03-28T17:58:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 -- start start 2024-03-28T17:58:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 0x7f65481616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65481542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480cd470 con 0x7f65480d0980 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654806b540 con 0x7f65480d05a0 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.715+0000 7f654f600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480cd5f0 con 0x7f65480ceb00 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65481542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f6547e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 0x7f65481616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f6547e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 0x7f65481616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45006/0 (socket says 172.21.15.62:45006) 2024-03-28T17:58:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:36190/0 (socket says 172.21.15.62:36190) 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f6547e00640 1 -- 172.21.15.62:0/3566749570 learned_addr learned my addr 172.21.15.62:0/3566749570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 -- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 msgr2=0x7f65481616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 0x7f65481616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 -- 172.21.15.62:0/3566749570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 msgr2=0x7f65481542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65481542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.716+0000 7f654cc00640 1 -- 172.21.15.62:0/3566749570 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480c58d0 con 0x7f65480ceb00 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f654cc00640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f653c00c390 tx=0x7f653c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c017070 con 0x7f65480ceb00 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c0052d0 con 0x7f65480ceb00 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c003830 con 0x7f65480ceb00 2024-03-28T17:58:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65480c5bb0 con 0x7f65480ceb00 2024-03-28T17:58:32.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65480c6170 con 0x7f65480ceb00 2024-03-28T17:58:32.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f653c01a070 con 0x7f65480ceb00 2024-03-28T17:58:32.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.717+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f653c011590 con 0x7f65480ceb00 2024-03-28T17:58:32.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.718+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65480680c0 con 0x7f65480ceb00 2024-03-28T17:58:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.722+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f653c021020 con 0x7f65480ceb00 2024-03-28T17:58:32.742 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:32.742+0000 7fa523c45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:58:32.762 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:32.762+0000 7f4054643040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:58:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.788+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65480b2c20 con 0x7f65480ceb00 2024-03-28T17:58:32.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.789+0000 7f6544c00640 1 -- 172.21.15.62:0/3566749570 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f653c011b30 con 0x7f65480ceb00 2024-03-28T17:58:32.790 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:32.790 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:32.409101+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 msgr2=0x7f6548151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f653c00c390 tx=0x7f653c00c850 comp rx=0 tx=0).stop 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 shutdown_connections 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65480d0980 0x7f65481542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65480d05a0 0x7f65481616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 --2- 172.21.15.62:0/3566749570 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f65480ceb00 0x7f6548151b70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 >> 172.21.15.62:0/3566749570 conn(0x7f65480b38e0 msgr2=0x7f65480b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 shutdown_connections 2024-03-28T17:58:32.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:32.790+0000 7f654f600640 1 -- 172.21.15.62:0/3566749570 wait complete. 2024-03-28T17:58:32.807 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-28T17:58:32.807 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-28T17:58:33.145 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-28T17:58:33.145+0000 7f2a0860d640 -1 Falling back to public interface 2024-03-28T17:58:33.151 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-28T17:58:33.152+0000 7f784c3bb640 -1 Falling back to public interface 2024-03-28T17:58:33.152 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T17:58:33.152+0000 7faa37415640 -1 Falling back to public interface 2024-03-28T17:58:33.156 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T17:58:33.156+0000 7fab94e68640 -1 Falling back to public interface 2024-03-28T17:58:33.159 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-28T17:58:33.160+0000 7fb6e8549640 -1 Falling back to public interface 2024-03-28T17:58:33.176 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T17:58:33.177+0000 7fd79938c640 -1 Falling back to public interface 2024-03-28T17:58:33.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-28T17:58:33.188+0000 7fcd588b3640 -1 Falling back to public interface 2024-03-28T17:58:33.189 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T17:58:33.190+0000 7fb88ee11640 -1 Falling back to public interface 2024-03-28T17:58:33.614 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:33.777 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:33.777+0000 7fa523c45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:58:33.782 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.782+0000 7fae9b600640 1 Processor -- start 2024-03-28T17:58:33.782 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.783+0000 7fae9b600640 1 -- start start 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae9b600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9406cfb0 0x7fae941485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae9b600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae94162db0 con 0x7fae9415c920 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae9b600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae94163090 con 0x7fae94070e00 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae9b600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae94148b20 con 0x7fae9406cfb0 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae98c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fae9415c920 0x7fae9406c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38108/0 (socket says 172.21.15.62:38108) 2024-03-28T17:58:33.784 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae98c00640 1 -- 172.21.15.62:0/1808747339 learned_addr learned my addr 172.21.15.62:0/1808747339 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:33.785 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae93e00640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9406cfb0 0x7fae941485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.784+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116235385 0 0) 0x7fae94162db0 con 0x7fae9415c920 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c003520 con 0x7fae9415c920 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188188642 0 0) 0x7fae94163090 con 0x7fae94070e00 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94162db0 con 0x7fae94070e00 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496975354 0 0) 0x7fae7c003520 con 0x7fae9415c920 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9406cfb0 msgr2=0x7fae941485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:33.787 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9406cfb0 0x7fae941485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 >> v1:172.21.15.81:6789/0 conn(0x7fae94070e00 legacy=0x7fae940712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94163370 con 0x7fae9415c920 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fae88001670 con 0x7fae9415c920 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 >> v1:172.21.15.62:6789/0 conn(0x7fae9415c920 legacy=0x7fae9406c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae7c003b70 0x7fae7c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c004800 0x7fae7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c0051c0 0x7fae7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94148b20 con 0x7fae7c0051c0 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00c090 con 0x7fae7c004800 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00c290 con 0x7fae7c003b70 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae93e00640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9406cfb0 0x7fae941485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae99600640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c0051c0 0x7fae7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.785+0000 7fae98c00640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae7c003b70 0x7fae7c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.788 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae93e00640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c004800 0x7fae7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae99600640 1 -- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae7c003b70 msgr2=0x7fae7c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae99600640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae7c003b70 0x7fae7c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae99600640 1 -- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c004800 msgr2=0x7fae7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae99600640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c004800 0x7fae7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.786+0000 7fae99600640 1 -- 172.21.15.62:0/1808747339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c003520 con 0x7fae7c0051c0 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.787+0000 7fae99600640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c0051c0 0x7fae7c00b9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae8c001fd0 tx=0x7fae7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab6d6a248af8f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.787+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c001bf0 con 0x7fae7c0051c0 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.787+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c003b70 con 0x7fae7c0051c0 2024-03-28T17:58:33.789 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.787+0000 7fae93400640 1 -- 172.21.15.62:0/1808747339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0050d0 con 0x7fae7c0051c0 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 -- 172.21.15.62:0/1808747339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c0051c0 msgr2=0x7fae7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 --2- 172.21.15.62:0/1808747339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c0051c0 0x7fae7c00b9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae8c001fd0 tx=0x7fae7c005830 comp rx=0 tx=0).stop 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae99600640 1 -- 172.21.15.62:0/1808747339 reap_dead start 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 -- 172.21.15.62:0/1808747339 shutdown_connections 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 -- 172.21.15.62:0/1808747339 >> 172.21.15.62:0/1808747339 conn(0x7fae940b16f0 msgr2=0x7fae940b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:33.791 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 -- 172.21.15.62:0/1808747339 shutdown_connections 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.788+0000 7fae9b600640 1 -- 172.21.15.62:0/1808747339 wait complete. 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.789+0000 7fae9b600640 1 Processor -- start 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.790+0000 7fae9b600640 1 -- start start 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.790+0000 7fae9b600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c0051c0 0x7fae94161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.790+0000 7fae9b600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae9b600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9415f6d0 0x7fae940b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae9b600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94162a10 con 0x7fae94161980 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae9b600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94163930 con 0x7fae7c0051c0 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae9b600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae94162b90 con 0x7fae9415f6d0 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52794/0 (socket says 172.21.15.62:52794) 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 -- 172.21.15.62:0/1117602373 learned_addr learned my addr 172.21.15.62:0/1117602373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:33.792 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae98c00640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c0051c0 0x7fae94161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae99600640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9415f6d0 0x7fae940b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 -- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9415f6d0 msgr2=0x7fae940b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9415f6d0 0x7fae940b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 -- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c0051c0 msgr2=0x7fae94161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c0051c0 0x7fae94161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.793 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.791+0000 7fae93e00640 1 -- 172.21.15.62:0/1117602373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9415ffe0 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae93e00640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae80011830 tx=0x7fae80011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000f9c0 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8000fb60 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80015460 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae941602c0 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.792+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae94160880 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.793+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fae80015600 con 0x7fae94161980 2024-03-28T17:58:33.794 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.793+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fae8001d400 con 0x7fae94161980 2024-03-28T17:58:33.795 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.793+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae941454f0 con 0x7fae94161980 2024-03-28T17:58:33.796 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.796+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fae941454f0 con 0x7fae94161980 2024-03-28T17:58:33.840 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:33.840+0000 7f4054643040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:58:33.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.874+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae94072d60 con 0x7fae94161980 2024-03-28T17:58:33.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.874+0000 7fae90c00640 1 -- 172.21.15.62:0/1117602373 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fae8002b020 con 0x7fae94161980 2024-03-28T17:58:33.875 INFO:teuthology.misc.health.smithi062.stdout: 2024-03-28T17:58:33.875 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":10,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:32.409101+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 msgr2=0x7fae9415b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae80011830 tx=0x7fae80011cf0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 shutdown_connections 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fae9415f6d0 0x7fae940b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae94161980 0x7fae9415b750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 --2- 172.21.15.62:0/1117602373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae7c0051c0 0x7fae94161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:33.876 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 >> 172.21.15.62:0/1117602373 conn(0x7fae940b16f0 msgr2=0x7fae94144c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:33.877 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 shutdown_connections 2024-03-28T17:58:33.877 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:33.876+0000 7fae9b600640 1 -- 172.21.15.62:0/1117602373 wait complete. 2024-03-28T17:58:33.888 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-28T17:58:34.091 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:34.092+0000 7fa523c45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:58:34.150 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:34.151+0000 7fa523c45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:58:34.175 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:34.175+0000 7f4054643040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:58:34.238 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:34.238+0000 7f4054643040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:58:34.281 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:34.282+0000 7fa523c45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:58:34.377 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-03-28T17:58:34.377+0000 7f4054643040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:58:35.327 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:35.327+0000 7fa4c4600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-28T17:58:35.327 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-03-28T17:58:35.328+0000 7fa4c4600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-28T17:58:36.481 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-28T17:58:36.482+0000 7f2a0860d640 -1 osd.5 0 log_to_monitors true 2024-03-28T17:58:36.509 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T17:58:36.509+0000 7fd79938c640 -1 osd.2 0 log_to_monitors true 2024-03-28T17:58:36.700 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-28T17:58:36.700+0000 7f784c3bb640 -1 osd.6 0 log_to_monitors true 2024-03-28T17:58:36.726 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-28T17:58:36.727+0000 7fb6e8549640 -1 osd.7 0 log_to_monitors true 2024-03-28T17:58:36.744 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T17:58:36.744+0000 7fab94e68640 -1 osd.0 0 log_to_monitors true 2024-03-28T17:58:36.762 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-28T17:58:36.762+0000 7fcd588b3640 -1 osd.4 0 log_to_monitors true 2024-03-28T17:58:36.969 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T17:58:36.969+0000 7faa37415640 -1 osd.1 0 log_to_monitors true 2024-03-28T17:58:37.013 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T17:58:37.012+0000 7fb88ee11640 -1 osd.3 0 log_to_monitors true 2024-03-28T17:58:40.563 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-28T17:58:40.563+0000 7fcd53800640 -1 osd.4 0 waiting for initial osdmap 2024-03-28T17:58:40.570 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-28T17:58:40.570+0000 7fcd49800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.666 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-28T17:58:40.666+0000 7fb6e3400640 -1 osd.7 0 waiting for initial osdmap 2024-03-28T17:58:40.674 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-28T17:58:40.675+0000 7fb6d9400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.695 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:40.712 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T17:58:40.711+0000 7fb889c00640 -1 osd.3 0 waiting for initial osdmap 2024-03-28T17:58:40.720 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T17:58:40.720+0000 7fb87fc00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.736 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T17:58:40.736+0000 7faa31c00640 -1 osd.1 0 waiting for initial osdmap 2024-03-28T17:58:40.745 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-28T17:58:40.746+0000 7f7847200640 -1 osd.6 0 waiting for initial osdmap 2024-03-28T17:58:40.747 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T17:58:40.747+0000 7faa27c00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.748 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T17:58:40.748+0000 7fab8fe00640 -1 osd.0 0 waiting for initial osdmap 2024-03-28T17:58:40.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-28T17:58:40.751+0000 7f783d200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.751 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-28T17:58:40.752+0000 7f2a03800640 -1 osd.5 0 waiting for initial osdmap 2024-03-28T17:58:40.756 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T17:58:40.756+0000 7fab85e00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.761 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-28T17:58:40.761+0000 7f29f8e00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.764 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T17:58:40.764+0000 7fd794200640 -1 osd.2 0 waiting for initial osdmap 2024-03-28T17:58:40.772 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T17:58:40.772+0000 7fd78a200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:40.862 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.862+0000 7f6c3a000640 1 Processor -- start 2024-03-28T17:58:40.862 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.862+0000 7f6c3a000640 1 -- start start 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340d0550 0x7f6c340d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c3a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c340ce790 0x7f6c340cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c3a000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c341618d0 con 0x7f6c341614f0 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c3a000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340d1bd0 con 0x7f6c340ce790 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c3a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340cf5c0 con 0x7f6c340d0550 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c340ce790 0x7f6c340cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c340ce790 0x7f6c340cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40272/0 (socket says 172.21.15.62:40272) 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 -- 172.21.15.62:0/1250702175 learned_addr learned my addr 172.21.15.62:0/1250702175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:40.865 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c38c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6c341614f0 0x7f6c340b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51596/0 (socket says 172.21.15.62:51596) 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c2be00640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340d0550 0x7f6c340d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 -- 172.21.15.62:0/1250702175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340d0550 msgr2=0x7f6c340d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340d0550 0x7f6c340d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 -- 172.21.15.62:0/1250702175 >> v1:172.21.15.62:6789/0 conn(0x7f6c341614f0 legacy=0x7f6c340b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c39600640 1 -- 172.21.15.62:0/1250702175 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340cf8a0 con 0x7f6c340ce790 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.863+0000 7f6c33e00640 1 -- 172.21.15.62:0/1250702175 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813476455 0 0) 0x7f6c341618d0 con 0x7f6c341614f0 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.864+0000 7f6c39600640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c340ce790 0x7f6c340cec00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e50 tx=0x7f6c2c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998a2b9b164354d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c33e00640 1 -- 172.21.15.62:0/1250702175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c02cb00 con 0x7f6c340ce790 2024-03-28T17:58:40.866 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c33e00640 1 -- 172.21.15.62:0/1250702175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c02cca0 con 0x7f6c340ce790 2024-03-28T17:58:40.867 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c33e00640 1 -- 172.21.15.62:0/1250702175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c034610 con 0x7f6c340ce790 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 -- 172.21.15.62:0/1250702175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c340ce790 msgr2=0x7f6c340cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c340ce790 0x7f6c340cec00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e50 tx=0x7f6c2c02bf60 comp rx=0 tx=0).stop 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 -- 172.21.15.62:0/1250702175 shutdown_connections 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c340ce790 0x7f6c340cec00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1250702175 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340d0550 0x7f6c340d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 -- 172.21.15.62:0/1250702175 >> 172.21.15.62:0/1250702175 conn(0x7f6c3409cf20 msgr2=0x7f6c340d3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 -- 172.21.15.62:0/1250702175 shutdown_connections 2024-03-28T17:58:40.870 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.865+0000 7f6c3a000640 1 -- 172.21.15.62:0/1250702175 wait complete. 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.867+0000 7f6c3a000640 1 Processor -- start 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.867+0000 7f6c3a000640 1 -- start start 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340ce790 0x7f6c34152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c340d0550 0x7f6c34156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340cf140 con 0x7f6c341614f0 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340d1860 con 0x7f6c340d0550 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c3a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340cf2c0 con 0x7f6c340ce790 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c39600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c39600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40276/0 (socket says 172.21.15.62:40276) 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.868+0000 7f6c39600640 1 -- 172.21.15.62:0/1383168167 learned_addr learned my addr 172.21.15.62:0/1383168167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:40.871 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c2be00640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c340d0550 0x7f6c34156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c38c00640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340ce790 0x7f6c34152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 -- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340ce790 msgr2=0x7f6c34152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340ce790 0x7f6c34152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 -- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c340d0550 msgr2=0x7f6c34156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c340d0550 0x7f6c34156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 -- 172.21.15.62:0/1383168167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34152c90 con 0x7f6c341614f0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c39600640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e50 tx=0x7f6c2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c043020 con 0x7f6c341614f0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c02ce50 con 0x7f6c341614f0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c009510 con 0x7f6c341614f0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c34152f70 con 0x7f6c341614f0 2024-03-28T17:58:40.872 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.869+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c34153530 con 0x7f6c341614f0 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.871+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c340752c0 con 0x7f6c341614f0 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.871+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f6c2c046020 con 0x7f6c341614f0 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.872+0000 7f6c31600640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 0x7f6c1006e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.872+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f6c2c0ac210 con 0x7f6c341614f0 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.873+0000 7f6c38c00640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 0x7f6c1006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:40.874 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.874+0000 7f6c38c00640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 0x7f6c1006e000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000f60 tx=0x7f6c24007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:40.877 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:40.877+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6c2c04d030 con 0x7f6c341614f0 2024-03-28T17:58:41.069 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.068+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c340d1270 con 0x7f6c341614f0 2024-03-28T17:58:41.069 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.069+0000 7f6c31600640 1 -- 172.21.15.62:0/1383168167 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f6c2c07bb60 con 0x7f6c341614f0 2024-03-28T17:58:41.070 INFO:teuthology.misc.health.smithi062.stdout: 2024-03-28T17:58:41.070 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":12,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:38.324013+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:41.072 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.071+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 msgr2=0x7f6c1006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:41.072 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.071+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 0x7f6c1006e000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000f60 tx=0x7f6c24007b50 comp rx=0 tx=0).stop 2024-03-28T17:58:41.072 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 msgr2=0x7f6c340cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:41.072 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e50 tx=0x7f6c2c004a40 comp rx=0 tx=0).stop 2024-03-28T17:58:41.072 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 shutdown_connections 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c1006bb40 0x7f6c1006e000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c341614f0 0x7f6c340cca20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c340d0550 0x7f6c34156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 --2- 172.21.15.62:0/1383168167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c340ce790 0x7f6c34152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 >> 172.21.15.62:0/1383168167 conn(0x7f6c3409cf20 msgr2=0x7f6c3406cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 shutdown_connections 2024-03-28T17:58:41.073 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:41.072+0000 7f6c3a000640 1 -- 172.21.15.62:0/1383168167 wait complete. 2024-03-28T17:58:41.085 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-28T17:58:43.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:58:43.859 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:58:47.892 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:48.087 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.087+0000 7fbde9400640 1 Processor -- start 2024-03-28T17:58:48.087 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.087+0000 7fbde9400640 1 -- start start 2024-03-28T17:58:48.089 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde9400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 0x7fbde40b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbde40ce840 0x7fbde40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde9400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde406d820 con 0x7fbde4145380 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde9400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde406db00 con 0x7fbde40ce840 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde9400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde406dde0 con 0x7fbde4145760 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbde40ce840 0x7fbde40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.089+0000 7fbde2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbde4145380 0x7fbde406e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58300/0 (socket says 172.21.15.62:58300) 2024-03-28T17:58:48.090 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde2800640 1 -- 172.21.15.62:0/1317833593 learned_addr learned my addr 172.21.15.62:0/1317833593 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde1e00640 1 --2- 172.21.15.62:0/1317833593 >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 0x7fbde40b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde3200640 1 -- 172.21.15.62:0/1317833593 >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 msgr2=0x7fbde40b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde3200640 1 --2- 172.21.15.62:0/1317833593 >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 0x7fbde40b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde3200640 1 -- 172.21.15.62:0/1317833593 >> v1:172.21.15.62:6789/0 conn(0x7fbde4145380 legacy=0x7fbde406e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde3200640 1 -- 172.21.15.62:0/1317833593 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40cf1b0 con 0x7fbde40ce840 2024-03-28T17:58:48.091 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.090+0000 7fbde1e00640 1 --2- 172.21.15.62:0/1317833593 >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 0x7fbde40b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:48.092 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.091+0000 7fbde3200640 1 --2- 172.21.15.62:0/1317833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde40ce840 0x7fbde40cec70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbde4072fa0 tx=0x7fbddc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b48b2a1166eb203 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.092 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.091+0000 7fbde1400640 1 -- 172.21.15.62:0/1317833593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0315a0 con 0x7fbde40ce840 2024-03-28T17:58:48.092 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.091+0000 7fbde1400640 1 -- 172.21.15.62:0/1317833593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc037070 con 0x7fbde40ce840 2024-03-28T17:58:48.092 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde1400640 1 -- 172.21.15.62:0/1317833593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc031960 con 0x7fbde40ce840 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 -- 172.21.15.62:0/1317833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde40ce840 msgr2=0x7fbde40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 --2- 172.21.15.62:0/1317833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde40ce840 0x7fbde40cec70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbde4072fa0 tx=0x7fbddc02b840 comp rx=0 tx=0).stop 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 -- 172.21.15.62:0/1317833593 shutdown_connections 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 --2- 172.21.15.62:0/1317833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde40ce840 0x7fbde40cec70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 --2- 172.21.15.62:0/1317833593 >> v2:172.21.15.81:3300/0 conn(0x7fbde4145760 0x7fbde40b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 -- 172.21.15.62:0/1317833593 >> 172.21.15.62:0/1317833593 conn(0x7fbde40b16f0 msgr2=0x7fbde40b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 -- 172.21.15.62:0/1317833593 shutdown_connections 2024-03-28T17:58:48.093 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.092+0000 7fbde9400640 1 -- 172.21.15.62:0/1317833593 wait complete. 2024-03-28T17:58:48.094 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.094+0000 7fbde9400640 1 Processor -- start 2024-03-28T17:58:48.094 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.094+0000 7fbde9400640 1 -- start start 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbde40ce840 0x7fbde4143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbde4145380 0x7fbde41f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b30b0 con 0x7fbde4145760 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b2f30 con 0x7fbde4145380 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde9400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde415c2f0 con 0x7fbde40ce840 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52436/0 (socket says 172.21.15.62:52436) 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.095+0000 7fbde3200640 1 -- 172.21.15.62:0/3946549643 learned_addr learned my addr 172.21.15.62:0/3946549643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde1e00640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbde4145380 0x7fbde41f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde2800640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbde40ce840 0x7fbde4143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 -- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbde40ce840 msgr2=0x7fbde4143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.097 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbde40ce840 0x7fbde4143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 -- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbde4145380 msgr2=0x7fbde41f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbde4145380 0x7fbde41f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 -- 172.21.15.62:0/3946549643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4141220 con 0x7fbde4145760 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbde3200640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009890 tx=0x7fbddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.096+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc032840 con 0x7fbde4145760 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.097+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc0329e0 con 0x7fbde4145760 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.097+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc032db0 con 0x7fbde4145760 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.097+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde41f4210 con 0x7fbde4145760 2024-03-28T17:58:48.098 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.097+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde41f47d0 con 0x7fbde4145760 2024-03-28T17:58:48.099 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.098+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4140f90 con 0x7fbde4145760 2024-03-28T17:58:48.099 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.099+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbddc04e3f0 con 0x7fbde4145760 2024-03-28T17:58:48.100 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.100+0000 7fbddaa00640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 0x7fbdc006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.100 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.100+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbddc02f070 con 0x7fbde4145760 2024-03-28T17:58:48.100 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.100+0000 7fbde2800640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 0x7fbdc006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.101 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.102+0000 7fbde2800640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 0x7fbdc006e0e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde4072970 tx=0x7fbdd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.105 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.106+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbddc040070 con 0x7fbde4145760 2024-03-28T17:58:48.313 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.313+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde404f400 con 0x7fbde4145760 2024-03-28T17:58:48.314 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.314+0000 7fbddaa00640 1 -- 172.21.15.62:0/3946549643 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12836 (secure 0 0 0) 0x7fbddc07b460 con 0x7fbde4145760 2024-03-28T17:58:48.315 INFO:teuthology.misc.health.smithi062.stdout: 2024-03-28T17:58:48.315 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":16,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:44.631633+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:48.317 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 msgr2=0x7fbdc006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.317 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 0x7fbdc006e0e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde4072970 tx=0x7fbdd4009040 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 msgr2=0x7fbde41409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009890 tx=0x7fbddc0046c0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 shutdown_connections 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbdc006bc20 0x7fbdc006e0e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde4145760 0x7fbde41409f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbde4145380 0x7fbde41f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 --2- 172.21.15.62:0/3946549643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbde40ce840 0x7fbde4143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 >> 172.21.15.62:0/3946549643 conn(0x7fbde40b16f0 msgr2=0x7fbde40ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.317+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 shutdown_connections 2024-03-28T17:58:48.318 INFO:teuthology.misc.health.smithi062.stderr:2024-03-28T17:58:48.318+0000 7fbde9400640 1 -- 172.21.15.62:0/3946549643 wait complete. 2024-03-28T17:58:48.333 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-28T17:58:48.333 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-28T17:58:48.333 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T17:58:48.333 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T17:58:48.339 INFO:tasks.ceph.mds.a:Started 2024-03-28T17:58:48.340 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T17:58:48.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T17:58:48.347 INFO:tasks.ceph.mds.c:Started 2024-03-28T17:58:48.347 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T17:58:48.347 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T17:58:48.354 INFO:tasks.ceph.mds.b:Started 2024-03-28T17:58:48.355 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T17:58:48.355 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T17:58:48.362 INFO:tasks.ceph.mds.d:Started 2024-03-28T17:58:48.362 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-28T17:58:48.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:48.494 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-03-28T17:58:48.501 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-03-28T17:58:48.517 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-28T17:58:48.518 INFO:tasks.ceph.mds.b.smithi081.stdout:starting mds.b at 2024-03-28T17:58:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.558+0000 7fbc9f200640 1 Processor -- start 2024-03-28T17:58:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.558+0000 7fbc9f200640 1 -- start start 2024-03-28T17:58:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.559+0000 7fbc9f200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbca015dd60 0x7fbca00b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.559+0000 7fbc9f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca00b99f0 0x7fbca00b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.559+0000 7fbc9f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca00b6040 con 0x7fbca00b5c30 2024-03-28T17:58:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.559+0000 7fbc9f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca00b6c30 con 0x7fbca00b99f0 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.559+0000 7fbc9f200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca00b6e60 con 0x7fbca015dd60 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca00b99f0 0x7fbca00b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc9e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbca00b5c30 0x7fbca0161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58306/0 (socket says 172.21.15.62:58306) 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca00b99f0 0x7fbca00b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52468/0 (socket says 172.21.15.62:52468) 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc9e800640 1 -- 172.21.15.62:0/280984424 learned_addr learned my addr 172.21.15.62:0/280984424 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc9de00640 1 --2- 172.21.15.62:0/280984424 >> v2:172.21.15.81:3300/0 conn(0x7fbca015dd60 0x7fbca00b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 -- 172.21.15.62:0/280984424 >> v2:172.21.15.81:3300/0 conn(0x7fbca015dd60 msgr2=0x7fbca00b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 --2- 172.21.15.62:0/280984424 >> v2:172.21.15.81:3300/0 conn(0x7fbca015dd60 0x7fbca00b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 -- 172.21.15.62:0/280984424 >> v1:172.21.15.62:6789/0 conn(0x7fbca00b5c30 legacy=0x7fbca0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc9d400640 1 -- 172.21.15.62:0/280984424 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853586766 0 0) 0x7fbc98001490 con 0x7fbca00b5c30 2024-03-28T17:58:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.560+0000 7fbc97e00640 1 -- 172.21.15.62:0/280984424 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca00b7100 con 0x7fbca00b99f0 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.561+0000 7fbc97e00640 1 --2- 172.21.15.62:0/280984424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca00b56f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbca00d1570 tx=0x7fbc8802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49980f7f95e40b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9d400640 1 -- 172.21.15.62:0/280984424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88034070 con 0x7fbca00b99f0 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9d400640 1 -- 172.21.15.62:0/280984424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88030610 con 0x7fbca00b99f0 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9d400640 1 -- 172.21.15.62:0/280984424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880309e0 con 0x7fbca00b99f0 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 -- 172.21.15.62:0/280984424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 msgr2=0x7fbca00b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 --2- 172.21.15.62:0/280984424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca00b56f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbca00d1570 tx=0x7fbc8802b960 comp rx=0 tx=0).stop 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 -- 172.21.15.62:0/280984424 shutdown_connections 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 --2- 172.21.15.62:0/280984424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca00b56f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 --2- 172.21.15.62:0/280984424 >> v2:172.21.15.81:3300/0 conn(0x7fbca015dd60 0x7fbca00b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.562+0000 7fbc9f200640 1 -- 172.21.15.62:0/280984424 >> 172.21.15.62:0/280984424 conn(0x7fbca0071ae0 msgr2=0x7fbca0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.563+0000 7fbc9f200640 1 -- 172.21.15.62:0/280984424 shutdown_connections 2024-03-28T17:58:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.563+0000 7fbc9f200640 1 -- 172.21.15.62:0/280984424 wait complete. 2024-03-28T17:58:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.564+0000 7fbc9f200640 1 Processor -- start 2024-03-28T17:58:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.564+0000 7fbc9f200640 1 -- start start 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 0x7fbca01594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbca015dd60 0x7fbca00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00b79a0 con 0x7fbca00b99f0 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00b76c0 con 0x7fbca00b5c30 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.565+0000 7fbc9f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca015e140 con 0x7fbca015dd60 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52480/0 (socket says 172.21.15.62:52480) 2024-03-28T17:58:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 -- 172.21.15.62:0/3639284481 learned_addr learned my addr 172.21.15.62:0/3639284481 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc9de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 0x7fbca01594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc9e800640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbca015dd60 0x7fbca00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 -- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbca015dd60 msgr2=0x7fbca00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbca015dd60 0x7fbca00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 -- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 msgr2=0x7fbca01594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 0x7fbca01594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 -- 172.21.15.62:0/3639284481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0160340 con 0x7fbca00b99f0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc9de00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 0x7fbca01594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.566+0000 7fbc97e00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc880096f0 tx=0x7fbc880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.567+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88034040 con 0x7fbca00b99f0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.567+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88030410 con 0x7fbca00b99f0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.567+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88039c70 con 0x7fbca00b99f0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.567+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca00c8490 con 0x7fbca00b99f0 2024-03-28T17:58:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.567+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca00c8a50 con 0x7fbca00b99f0 2024-03-28T17:58:48.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.568+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca0072b90 con 0x7fbca00b99f0 2024-03-28T17:58:48.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.569+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbc88037070 con 0x7fbca00b99f0 2024-03-28T17:58:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.570+0000 7fbc96000640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 0x7fbc7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.570+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc880ac5f0 con 0x7fbca00b99f0 2024-03-28T17:58:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.570+0000 7fbc9de00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 0x7fbc7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.571+0000 7fbc9de00640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 0x7fbc7c06e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009e90 tx=0x7fbc90009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:48.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.575+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbc8807ad40 con 0x7fbca00b99f0 2024-03-28T17:58:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.776+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca01e7790 con 0x7fbca00b99f0 2024-03-28T17:58:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.777+0000 7fbc96000640 1 -- 172.21.15.62:0/3639284481 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12836 (secure 0 0 0) 0x7fbc8807ede0 con 0x7fbca00b99f0 2024-03-28T17:58:48.777 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:48.777 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:44.631633+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.779+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 msgr2=0x7fbc7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.779+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 0x7fbc7c06e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009e90 tx=0x7fbc90009190 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.779+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 msgr2=0x7fbca01e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.779+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc880096f0 tx=0x7fbc880046c0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 shutdown_connections 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc7c06bbd0 0x7fbc7c06e090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbca015dd60 0x7fbca00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca00b99f0 0x7fbca01e9910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 --2- 172.21.15.62:0/3639284481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00b5c30 0x7fbca01594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 >> 172.21.15.62:0/3639284481 conn(0x7fbca0071ae0 msgr2=0x7fbca0162240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 shutdown_connections 2024-03-28T17:58:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:48.780+0000 7fbc9f200640 1 -- 172.21.15.62:0/3639284481 wait complete. 2024-03-28T17:58:48.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:58:48.799 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:58:49.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.027+0000 7f612be00640 1 Processor -- start 2024-03-28T17:58:49.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.027+0000 7f612be00640 1 -- start start 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.028+0000 7f612be00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f612c0b3210 0x7f612c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c161bb0 con 0x7f612c15e2c0 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612be00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c161e90 con 0x7f612c0cec50 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612be00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c162170 con 0x7f612c0b3210 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612a000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f612c15e2c0 0x7f612c0d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58318/0 (socket says 172.21.15.62:58318) 2024-03-28T17:58:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612a000640 1 -- 172.21.15.62:0/2786225060 learned_addr learned my addr 172.21.15.62:0/2786225060 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f612b400640 1 --2- 172.21.15.62:0/2786225060 >> v2:172.21.15.81:3300/0 conn(0x7f612c0b3210 0x7f612c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.029+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309372632 0 0) 0x7f612c161bb0 con 0x7f612c15e2c0 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.030+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6110003520 con 0x7f612c15e2c0 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.030+0000 7f612b400640 1 -- 172.21.15.62:0/2786225060 >> v1:172.21.15.81:6789/0 conn(0x7f612c0cec50 legacy=0x7f612c0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.030+0000 7f612b400640 1 -- 172.21.15.62:0/2786225060 >> v1:172.21.15.62:6789/0 conn(0x7f612c15e2c0 legacy=0x7f612c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.030+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43565650 0 0) 0x7f6110003520 con 0x7f612c15e2c0 2024-03-28T17:58:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.030+0000 7f612b400640 1 -- 172.21.15.62:0/2786225060 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c162450 con 0x7f612c0b3210 2024-03-28T17:58:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.031+0000 7f612b400640 1 --2- 172.21.15.62:0/2786225060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c0b3210 0x7f612c0ce570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df41f293486c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124039070 con 0x7f612c0b3210 2024-03-28T17:58:49.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612402c940 con 0x7f612c0b3210 2024-03-28T17:58:49.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f6129600640 1 -- 172.21.15.62:0/2786225060 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c001670 con 0x7f612c0b3210 2024-03-28T17:58:49.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f612be00640 1 -- 172.21.15.62:0/2786225060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c0b3210 msgr2=0x7f612c0ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f612be00640 1 --2- 172.21.15.62:0/2786225060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c0b3210 0x7f612c0ce570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612402be10 comp rx=0 tx=0).stop 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f612be00640 1 -- 172.21.15.62:0/2786225060 shutdown_connections 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.032+0000 7f612be00640 1 --2- 172.21.15.62:0/2786225060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c0b3210 0x7f612c0ce570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.033+0000 7f612be00640 1 -- 172.21.15.62:0/2786225060 >> 172.21.15.62:0/2786225060 conn(0x7f612c066fb0 msgr2=0x7f612c0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.033+0000 7f612be00640 1 -- 172.21.15.62:0/2786225060 shutdown_connections 2024-03-28T17:58:49.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.033+0000 7f612be00640 1 -- 172.21.15.62:0/2786225060 wait complete. 2024-03-28T17:58:49.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.035+0000 7f612be00640 1 Processor -- start 2024-03-28T17:58:49.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.035+0000 7f612be00640 1 -- start start 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.036+0000 7f612be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.036+0000 7f612be00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 0x7f612c1e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c1f7570 0x7f612c1f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0d26c0 con 0x7f612c0b3210 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612be00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c162a10 con 0x7f612c1f7570 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612be00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0d2840 con 0x7f612c0cec50 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52488/0 (socket says 172.21.15.62:52488) 2024-03-28T17:58:49.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 -- 172.21.15.62:0/2462636542 learned_addr learned my addr 172.21.15.62:0/2462636542 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612b400640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c1f7570 0x7f612c1f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612a000640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 0x7f612c1e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 -- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 msgr2=0x7f612c1e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 0x7f612c1e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 -- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c1f7570 msgr2=0x7f612c1f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c1f7570 0x7f612c1f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612aa00640 1 -- 172.21.15.62:0/2462636542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0c78c0 con 0x7f612c0b3210 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.037+0000 7f612a000640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 0x7f612c1e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f612aa00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f611400c450 tx=0x7f611400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61140052d0 con 0x7f612c0b3210 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114005470 con 0x7f612c0b3210 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114012870 con 0x7f612c0b3210 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c0c7ba0 con 0x7f612c0b3210 2024-03-28T17:58:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.038+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c0c8160 con 0x7f612c0b3210 2024-03-28T17:58:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.039+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c068ae0 con 0x7f612c0b3210 2024-03-28T17:58:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.040+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6114012a10 con 0x7f612c0b3210 2024-03-28T17:58:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.041+0000 7f6122a00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 0x7f610806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.041+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f611408a3c0 con 0x7f612c0b3210 2024-03-28T17:58:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.042+0000 7f612a000640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 0x7f610806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.043+0000 7f612a000640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 0x7f610806e0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f611c0011e0 tx=0x7f611c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.047+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f612c068ae0 con 0x7f612c0b3210 2024-03-28T17:58:49.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.244+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f612c0d2360 con 0x7f612c0b3210 2024-03-28T17:58:49.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.245+0000 7f6122a00640 1 -- 172.21.15.62:0/2462636542 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6114059160 con 0x7f612c0b3210 2024-03-28T17:58:49.245 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:58:49.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 msgr2=0x7f610806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 0x7f610806e0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f611c0011e0 tx=0x7f611c009040 comp rx=0 tx=0).stop 2024-03-28T17:58:49.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 msgr2=0x7f612c1e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f611400c450 tx=0x7f611400c910 comp rx=0 tx=0).stop 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 shutdown_connections 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f610806bc00 0x7f610806e0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f612c1f7570 0x7f612c1f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f612c0cec50 0x7f612c1e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 --2- 172.21.15.62:0/2462636542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c0b3210 0x7f612c1e78a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 >> 172.21.15.62:0/2462636542 conn(0x7f612c066fb0 msgr2=0x7f612c067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 shutdown_connections 2024-03-28T17:58:49.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.247+0000 7f612be00640 1 -- 172.21.15.62:0/2462636542 wait complete. 2024-03-28T17:58:49.264 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T17:58:49.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.480+0000 7f48ad400640 1 Processor -- start 2024-03-28T17:58:49.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.481+0000 7f48ad400640 1 -- start start 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48ad400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 0x7f48a80b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48ad400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48a80b5e00 0x7f48a80b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48ad400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a80b6e60 con 0x7f48a80b99f0 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48ad400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80b6c30 con 0x7f48a80b5e00 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48ad400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80b7100 con 0x7f48a815dd60 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.482+0000 7f48aca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48a80b5e00 0x7f48a80b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48a80b5e00 0x7f48a80b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52504/0 (socket says 172.21.15.62:52504) 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 -- 172.21.15.62:0/3711027912 learned_addr learned my addr 172.21.15.62:0/3711027912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:49.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48a7e00640 1 --2- 172.21.15.62:0/3711027912 >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 0x7f48a80b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 -- 172.21.15.62:0/3711027912 >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 msgr2=0x7f48a80b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 --2- 172.21.15.62:0/3711027912 >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 0x7f48a80b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 -- 172.21.15.62:0/3711027912 >> v1:172.21.15.62:6789/0 conn(0x7f48a80b99f0 legacy=0x7f48a80b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48aca00640 1 -- 172.21.15.62:0/3711027912 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80b73e0 con 0x7f48a80b5e00 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48a7e00640 1 --2- 172.21.15.62:0/3711027912 >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 0x7f48a80b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:49.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.483+0000 7f48a6a00640 1 -- 172.21.15.62:0/3711027912 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912105477 0 0) 0x7f48a80b6e60 con 0x7f48a80b99f0 2024-03-28T17:58:49.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48aca00640 1 --2- 172.21.15.62:0/3711027912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b5e00 0x7f48a80b7ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48a80d1570 tx=0x7f48a002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26048482cd1dbf84 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48a6a00640 1 -- 172.21.15.62:0/3711027912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a002f850 con 0x7f48a80b5e00 2024-03-28T17:58:49.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48a6a00640 1 -- 172.21.15.62:0/3711027912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a002f9f0 con 0x7f48a80b5e00 2024-03-28T17:58:49.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48a6a00640 1 -- 172.21.15.62:0/3711027912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a002fdc0 con 0x7f48a80b5e00 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48ad400640 1 -- 172.21.15.62:0/3711027912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b5e00 msgr2=0x7f48a80b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.485+0000 7f48ad400640 1 --2- 172.21.15.62:0/3711027912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b5e00 0x7f48a80b7ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48a80d1570 tx=0x7f48a002ba30 comp rx=0 tx=0).stop 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 -- 172.21.15.62:0/3711027912 shutdown_connections 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 --2- 172.21.15.62:0/3711027912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b5e00 0x7f48a80b7ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 --2- 172.21.15.62:0/3711027912 >> v2:172.21.15.81:3300/0 conn(0x7f48a815dd60 0x7f48a80b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 -- 172.21.15.62:0/3711027912 >> 172.21.15.62:0/3711027912 conn(0x7f48a8071ae0 msgr2=0x7f48a8071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 -- 172.21.15.62:0/3711027912 shutdown_connections 2024-03-28T17:58:49.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.486+0000 7f48ad400640 1 -- 172.21.15.62:0/3711027912 wait complete. 2024-03-28T17:58:49.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.487+0000 7f48ad400640 1 Processor -- start 2024-03-28T17:58:49.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.488+0000 7f48ad400640 1 -- start start 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.488+0000 7f48ad400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 0x7f48a80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.488+0000 7f48ad400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48ad400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 0x7f48a81f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48ad400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815e140 con 0x7f48a80b99f0 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48ad400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80b79a0 con 0x7f48a815dd60 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48ad400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80b6370 con 0x7f48a80b5e00 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52506/0 (socket says 172.21.15.62:52506) 2024-03-28T17:58:49.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 -- 172.21.15.62:0/212815899 learned_addr learned my addr 172.21.15.62:0/212815899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7e00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 0x7f48a80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48aca00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 0x7f48a81f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 -- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 msgr2=0x7f48a80ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 0x7f48a80ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 -- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 msgr2=0x7f48a81f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 0x7f48a81f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7400640 1 -- 172.21.15.62:0/212815899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a815f9d0 con 0x7f48a80b99f0 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48a7e00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 0x7f48a80ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.489+0000 7f48aca00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 0x7f48a81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f48a7400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f489800c890 tx=0x7f489800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898003b90 con 0x7f48a80b99f0 2024-03-28T17:58:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898003d70 con 0x7f48a80b99f0 2024-03-28T17:58:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a80c82e0 con 0x7f48a80b99f0 2024-03-28T17:58:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800aa90 con 0x7f48a80b99f0 2024-03-28T17:58:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.490+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a80c8860 con 0x7f48a80b99f0 2024-03-28T17:58:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.491+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8072b90 con 0x7f48a80b99f0 2024-03-28T17:58:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.492+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f489800ac30 con 0x7f48a80b99f0 2024-03-28T17:58:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.493+0000 7f489fe00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.493+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f489808a7a0 con 0x7f48a80b99f0 2024-03-28T17:58:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.493+0000 7f48a7e00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.494+0000 7f48a7e00640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 0x7f488406e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.498+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4898059690 con 0x7f48a80b99f0 2024-03-28T17:58:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.731+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a80d1840 con 0x7f48a80b99f0 2024-03-28T17:58:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.732+0000 7f489fe00640 1 -- 172.21.15.62:0/212815899 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f4898058f90 con 0x7f48a80b99f0 2024-03-28T17:58:49.733 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:49.733 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T17:58:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-03-28T17:58:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 msgr2=0x7f488406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 0x7f488406e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890008040 comp rx=0 tx=0).stop 2024-03-28T17:58:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 msgr2=0x7f48a81603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f489800c890 tx=0x7f489800cd50 comp rx=0 tx=0).stop 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 shutdown_connections 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48a815dd60 0x7f48a81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48a80b99f0 0x7f48a81603f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 --2- 172.21.15.62:0/212815899 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f48a80b5e00 0x7f48a80ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.735+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 >> 172.21.15.62:0/212815899 conn(0x7f48a8071ae0 msgr2=0x7f48a80b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.736+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 shutdown_connections 2024-03-28T17:58:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.736+0000 7f48ad400640 1 -- 172.21.15.62:0/212815899 wait complete. 2024-03-28T17:58:49.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:49.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.970+0000 7f15a9600640 1 Processor -- start 2024-03-28T17:58:49.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.970+0000 7f15a9600640 1 -- start start 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a9600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15a4071d90 0x7f15a4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a9600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a40b30d0 con 0x7f15a40ae370 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a9600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a40b33b0 con 0x7f15a40719b0 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a9600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40b3690 con 0x7f15a4071d90 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a2000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f15a40ae370 0x7f15a40ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58332/0 (socket says 172.21.15.62:58332) 2024-03-28T17:58:49.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a2000640 1 -- 172.21.15.62:0/1867959481 learned_addr learned my addr 172.21.15.62:0/1867959481 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a3400640 1 --2- 172.21.15.62:0/1867959481 >> v2:172.21.15.81:3300/0 conn(0x7f15a4071d90 0x7f15a4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855661392 0 0) 0x7f15a40b30d0 con 0x7f15a40ae370 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.972+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1588003520 con 0x7f15a40ae370 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358080976 0 0) 0x7f1588003520 con 0x7f15a40ae370 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 >> v2:172.21.15.81:3300/0 conn(0x7f15a4071d90 msgr2=0x7f15a4072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 --2- 172.21.15.62:0/1867959481 >> v2:172.21.15.81:3300/0 conn(0x7f15a4071d90 0x7f15a4072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 >> v1:172.21.15.81:6789/0 conn(0x7f15a40719b0 legacy=0x7f15a40b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40b3970 con 0x7f15a40ae370 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f1590001670 con 0x7f15a40ae370 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 >> v1:172.21.15.62:6789/0 conn(0x7f15a40ae370 legacy=0x7f15a40ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15880039b0 0x7f1588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 0x7f1588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15880051c0 0x7f158800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b3690 con 0x7f1588004800 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158800c060 con 0x7f15880039b0 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158800c260 con 0x7f15880051c0 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2000640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 0x7f1588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2a00640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15880039b0 0x7f1588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a3400640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15880051c0 0x7f158800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2000640 1 -- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15880051c0 msgr2=0x7f158800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2000640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15880051c0 0x7f158800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2000640 1 -- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15880039b0 msgr2=0x7f1588003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.973+0000 7f15a2000640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15880039b0 0x7f1588003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.974+0000 7f15a2000640 1 -- 172.21.15.62:0/1867959481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588003520 con 0x7f1588004800 2024-03-28T17:58:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.974+0000 7f15a3400640 1 -- 172.21.15.62:0/1867959481 reap_dead start 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a2000640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 0x7f1588004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f159000c240 tx=0x7f1588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d9df62f64dab91 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590007cf0 con 0x7f1588004800 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590030070 con 0x7f1588004800 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a1600640 1 -- 172.21.15.62:0/1867959481 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590005d70 con 0x7f1588004800 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a9600640 1 -- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 msgr2=0x7f1588004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a9600640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 0x7f1588004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f159000c240 tx=0x7f1588005830 comp rx=0 tx=0).stop 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a9600640 1 -- 172.21.15.62:0/1867959481 shutdown_connections 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a9600640 1 --2- 172.21.15.62:0/1867959481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1588004800 0x7f1588004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.975+0000 7f15a9600640 1 -- 172.21.15.62:0/1867959481 >> 172.21.15.62:0/1867959481 conn(0x7f15a4066fb0 msgr2=0x7f15a40673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.976+0000 7f15a9600640 1 -- 172.21.15.62:0/1867959481 shutdown_connections 2024-03-28T17:58:49.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.976+0000 7f15a9600640 1 -- 172.21.15.62:0/1867959481 wait complete. 2024-03-28T17:58:49.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.977+0000 7f15a9600640 1 Processor -- start 2024-03-28T17:58:49.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.977+0000 7f15a9600640 1 -- start start 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1588004800 0x7f15a40cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15a40b07e0 0x7f15a40b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4072380 con 0x7f15a414ec30 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4072200 con 0x7f1588004800 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a9600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4072500 con 0x7f15a40b07e0 2024-03-28T17:58:49.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.978+0000 7f15a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52520/0 (socket says 172.21.15.62:52520) 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 -- 172.21.15.62:0/379274625 learned_addr learned my addr 172.21.15.62:0/379274625 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 -- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15a40b07e0 msgr2=0x7f15a40b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2a00640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1588004800 0x7f15a40cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a3400640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15a40b07e0 0x7f15a40b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15a40b07e0 0x7f15a40b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 -- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1588004800 msgr2=0x7f15a40cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1588004800 0x7f15a40cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 -- 172.21.15.62:0/379274625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a414f0c0 con 0x7f15a414ec30 2024-03-28T17:58:49.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.979+0000 7f15a2000640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1590001970 tx=0x7f15a40b32a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.980+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590007cf0 con 0x7f15a414ec30 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.980+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159002cbd0 con 0x7f15a414ec30 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.980+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a414f3a0 con 0x7f15a414ec30 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.980+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590001c30 con 0x7f15a414ec30 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.980+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a414f960 con 0x7f15a414ec30 2024-03-28T17:58:49.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.981+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a40680c0 con 0x7f15a414ec30 2024-03-28T17:58:49.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.982+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1590007070 con 0x7f15a414ec30 2024-03-28T17:58:49.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.983+0000 7f159aa00640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 0x7f158406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:49.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.983+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15900abf20 con 0x7f15a414ec30 2024-03-28T17:58:49.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.983+0000 7f15a2a00640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 0x7f158406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:49.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.984+0000 7f15a2a00640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 0x7f158406e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1594007fd0 tx=0x7f1594009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:49.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:49.987+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f159007ad60 con 0x7f15a414ec30 2024-03-28T17:58:50.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.189+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a414cc50 con 0x7f15a414ec30 2024-03-28T17:58:50.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.191+0000 7f159aa00640 1 -- 172.21.15.62:0/379274625 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12836 (secure 0 0 0) 0x7f159007a660 con 0x7f15a414ec30 2024-03-28T17:58:50.191 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:50.191 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:44.631633+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 msgr2=0x7f158406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 0x7f158406e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1594007fd0 tx=0x7f1594009040 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 msgr2=0x7f15a414c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1590001970 tx=0x7f15a40b32a0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 shutdown_connections 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f158406bbd0 0x7f158406e090 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f15a40b07e0 0x7f15a40b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15a414ec30 0x7f15a414c470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 --2- 172.21.15.62:0/379274625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1588004800 0x7f15a40cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.193+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 >> 172.21.15.62:0/379274625 conn(0x7f15a4066fb0 msgr2=0x7f15a40ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:50.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.194+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 shutdown_connections 2024-03-28T17:58:50.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.194+0000 7f15a9600640 1 -- 172.21.15.62:0/379274625 wait complete. 2024-03-28T17:58:50.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:58:50.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:58:50.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.433+0000 7f9a85800640 1 Processor -- start 2024-03-28T17:58:50.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.434+0000 7f9a85800640 1 -- start start 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.435+0000 7f9a85800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a80072690 0x7f9a800cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.435+0000 7f9a85800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a800ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.435+0000 7f9a85800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8015d1b0 con 0x7f9a800ae720 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.435+0000 7f9a85800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a80070d50 con 0x7f9a80072690 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.435+0000 7f9a85800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80071030 con 0x7f9a80072b00 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7ea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9a800ae720 0x7f9a800aeb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58346/0 (socket says 172.21.15.62:58346) 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7ea00640 1 -- 172.21.15.62:0/4222950089 learned_addr learned my addr 172.21.15.62:0/4222950089 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:50.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7e000640 1 --2- 172.21.15.62:0/4222950089 >> v2:172.21.15.62:3300/0 conn(0x7f9a80072690 0x7f9a800cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7f400640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a800ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7d600640 1 -- 172.21.15.62:0/4222950089 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875978845 0 0) 0x7f9a8015d1b0 con 0x7f9a800ae720 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7d600640 1 -- 172.21.15.62:0/4222950089 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a64003520 con 0x7f9a800ae720 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7e000640 1 -- 172.21.15.62:0/4222950089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 msgr2=0x7f9a800ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7e000640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a800ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7e000640 1 -- 172.21.15.62:0/4222950089 >> v1:172.21.15.62:6789/0 conn(0x7f9a800ae720 legacy=0x7f9a800aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.436+0000 7f9a7e000640 1 -- 172.21.15.62:0/4222950089 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80071310 con 0x7f9a80072690 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.437+0000 7f9a7f400640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a800ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.438+0000 7f9a7e000640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a800cde50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a680097a0 tx=0x7f9a6802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dec945f565553e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.438+0000 7f9a7d600640 1 -- 172.21.15.62:0/4222950089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68035020 con 0x7f9a80072690 2024-03-28T17:58:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.438+0000 7f9a7d600640 1 -- 172.21.15.62:0/4222950089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68031810 con 0x7f9a80072690 2024-03-28T17:58:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.438+0000 7f9a7d600640 1 -- 172.21.15.62:0/4222950089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68031bc0 con 0x7f9a80072690 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 -- 172.21.15.62:0/4222950089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 msgr2=0x7f9a800cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a800cde50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a680097a0 tx=0x7f9a6802b8b0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 -- 172.21.15.62:0/4222950089 shutdown_connections 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a800ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 --2- 172.21.15.62:0/4222950089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a800cde50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 -- 172.21.15.62:0/4222950089 >> 172.21.15.62:0/4222950089 conn(0x7f9a800b1650 msgr2=0x7f9a800b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 -- 172.21.15.62:0/4222950089 shutdown_connections 2024-03-28T17:58:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.439+0000 7f9a85800640 1 -- 172.21.15.62:0/4222950089 wait complete. 2024-03-28T17:58:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.440+0000 7f9a85800640 1 Processor -- start 2024-03-28T17:58:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 -- start start 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a8014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a800ae720 0x7f9a800b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80070a40 con 0x7f9a80072690 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800718d0 con 0x7f9a800ae720 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.441+0000 7f9a85800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80153030 con 0x7f9a80072b00 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52536/0 (socket says 172.21.15.62:52536) 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 -- 172.21.15.62:0/742112964 learned_addr learned my addr 172.21.15.62:0/742112964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7e000640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a8014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7f400640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a800ae720 0x7f9a800b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 -- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 msgr2=0x7f9a8014d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a8014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 -- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a800ae720 msgr2=0x7f9a800b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a800ae720 0x7f9a800b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 -- 172.21.15.62:0/742112964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80153ee0 con 0x7f9a80072690 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.442+0000 7f9a7ea00640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c390 tx=0x7f9a7000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.443+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70017070 con 0x7f9a80072690 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.443+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a70005ce0 con 0x7f9a80072690 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.443+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a700072f0 con 0x7f9a80072690 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.443+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a801541c0 con 0x7f9a80072690 2024-03-28T17:58:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.443+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a80154780 con 0x7f9a80072690 2024-03-28T17:58:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.444+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a800680c0 con 0x7f9a80072690 2024-03-28T17:58:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.445+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9a7001a070 con 0x7f9a80072690 2024-03-28T17:58:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.446+0000 7f9a76a00640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 0x7f9a5c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.446+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a70089b80 con 0x7f9a80072690 2024-03-28T17:58:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.446+0000 7f9a7e000640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 0x7f9a5c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.447+0000 7f9a7e000640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 0x7f9a5c06e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a68008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.451+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9a700589b0 con 0x7f9a80072690 2024-03-28T17:58:50.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.652+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a800b2420 con 0x7f9a80072690 2024-03-28T17:58:50.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.652+0000 7f9a76a00640 1 -- 172.21.15.62:0/742112964 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a700582b0 con 0x7f9a80072690 2024-03-28T17:58:50.653 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.655+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 msgr2=0x7f9a5c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.655+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 0x7f9a5c06e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a68008030 comp rx=0 tx=0).stop 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.655+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 msgr2=0x7f9a801f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.655+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c390 tx=0x7f9a7000c850 comp rx=0 tx=0).stop 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 shutdown_connections 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9a5c06bbd0 0x7f9a5c06e090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a800ae720 0x7f9a800b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9a80072b00 0x7f9a8014d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 --2- 172.21.15.62:0/742112964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a80072690 0x7f9a801f3d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 >> 172.21.15.62:0/742112964 conn(0x7f9a800b1650 msgr2=0x7f9a800cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 shutdown_connections 2024-03-28T17:58:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.656+0000 7f9a85800640 1 -- 172.21.15.62:0/742112964 wait complete. 2024-03-28T17:58:50.676 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T17:58:50.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.897+0000 7fdc53200640 1 Processor -- start 2024-03-28T17:58:50.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.898+0000 7fdc53200640 1 -- start start 2024-03-28T17:58:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc53200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc4c15b200 0x7fdc4c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc53200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4c071380 con 0x7fdc4c070f80 2024-03-28T17:58:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc53200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4c15b810 con 0x7fdc4c155380 2024-03-28T17:58:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc53200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4c155760 con 0x7fdc4c15b200 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc51400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc4c15b200 0x7fdc4c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc52800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdc4c070f80 0x7fdc4c15cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58348/0 (socket says 172.21.15.62:58348) 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.899+0000 7fdc52800640 1 -- 172.21.15.62:0/3638012019 learned_addr learned my addr 172.21.15.62:0/3638012019 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.900+0000 7fdc51400640 1 -- 172.21.15.62:0/3638012019 >> v1:172.21.15.81:6789/0 conn(0x7fdc4c155380 legacy=0x7fdc4c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.900+0000 7fdc51400640 1 -- 172.21.15.62:0/3638012019 >> v1:172.21.15.62:6789/0 conn(0x7fdc4c070f80 legacy=0x7fdc4c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.900+0000 7fdc51400640 1 -- 172.21.15.62:0/3638012019 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c15baf0 con 0x7fdc4c15b200 2024-03-28T17:58:50.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.901+0000 7fdc51400640 1 --2- 172.21.15.62:0/3638012019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c070a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009790 tx=0x7fdc3c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee1541c9b6fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.901+0000 7fdc50a00640 1 -- 172.21.15.62:0/3638012019 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c035020 con 0x7fdc4c15b200 2024-03-28T17:58:50.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.901+0000 7fdc50a00640 1 -- 172.21.15.62:0/3638012019 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c031810 con 0x7fdc4c15b200 2024-03-28T17:58:50.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.901+0000 7fdc50a00640 1 -- 172.21.15.62:0/3638012019 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c031be0 con 0x7fdc4c15b200 2024-03-28T17:58:50.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 -- 172.21.15.62:0/3638012019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 msgr2=0x7fdc4c070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 --2- 172.21.15.62:0/3638012019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c070a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009790 tx=0x7fdc3c02b8b0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 -- 172.21.15.62:0/3638012019 shutdown_connections 2024-03-28T17:58:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 --2- 172.21.15.62:0/3638012019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c070a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 -- 172.21.15.62:0/3638012019 >> 172.21.15.62:0/3638012019 conn(0x7fdc4c0b1730 msgr2=0x7fdc4c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 -- 172.21.15.62:0/3638012019 shutdown_connections 2024-03-28T17:58:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.902+0000 7fdc53200640 1 -- 172.21.15.62:0/3638012019 wait complete. 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.904+0000 7fdc53200640 1 Processor -- start 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.904+0000 7fdc53200640 1 -- start start 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 0x7fdc4c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 0x7fdc4c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c15d250 con 0x7fdc4c15b200 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c15c0b0 con 0x7fdc4c155380 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc53200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c15d3d0 con 0x7fdc4c070f80 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc52800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc52800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52554/0 (socket says 172.21.15.62:52554) 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc52800640 1 -- 172.21.15.62:0/3347751305 learned_addr learned my addr 172.21.15.62:0/3347751305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc51400640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 0x7fdc4c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.905+0000 7fdc51e00640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 0x7fdc4c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 -- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 msgr2=0x7fdc4c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 0x7fdc4c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 -- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 msgr2=0x7fdc4c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 0x7fdc4c0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 -- 172.21.15.62:0/3347751305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c1e64f0 con 0x7fdc4c15b200 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc51400640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 0x7fdc4c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc51e00640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 0x7fdc4c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc52800640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800cbc0 tx=0x7fdc4800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc4c15b200 2024-03-28T17:58:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4800ad40 con 0x7fdc4c15b200 2024-03-28T17:58:50.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48012600 con 0x7fdc4c15b200 2024-03-28T17:58:50.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c1595e0 con 0x7fdc4c15b200 2024-03-28T17:58:50.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.906+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c159ba0 con 0x7fdc4c15b200 2024-03-28T17:58:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.907+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c1593f0 con 0x7fdc4c15b200 2024-03-28T17:58:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.908+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdc4801a070 con 0x7fdc4c15b200 2024-03-28T17:58:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.909+0000 7fdc46000640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 0x7fdc3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.909+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc4808a4b0 con 0x7fdc4c15b200 2024-03-28T17:58:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.910+0000 7fdc51e00640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 0x7fdc3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.911+0000 7fdc51e00640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 0x7fdc3006e090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c1f5860 tx=0x7fdc40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:50.914+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdc48058c00 con 0x7fdc4c15b200 2024-03-28T17:58:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.124+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdc4c073490 con 0x7fdc4c15b200 2024-03-28T17:58:51.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.125+0000 7fdc46000640 1 -- 172.21.15.62:0/3347751305 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdc4805cca0 con 0x7fdc4c15b200 2024-03-28T17:58:51.125 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T17:58:51.126 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 msgr2=0x7fdc3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 0x7fdc3006e090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c1f5860 tx=0x7fdc40009040 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 msgr2=0x7fdc4c1e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800cbc0 tx=0x7fdc4800a430 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 shutdown_connections 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc3006bbd0 0x7fdc3006e090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c15b200 0x7fdc4c1e58a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c155380 0x7fdc4c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 --2- 172.21.15.62:0/3347751305 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc4c070f80 0x7fdc4c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.128+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 >> 172.21.15.62:0/3347751305 conn(0x7fdc4c0b1730 msgr2=0x7fdc4c0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.129+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 shutdown_connections 2024-03-28T17:58:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.129+0000 7fdc53200640 1 -- 172.21.15.62:0/3347751305 wait complete. 2024-03-28T17:58:51.147 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T17:58:51.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T17:58:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.377+0000 7fa3ff800640 1 Processor -- start 2024-03-28T17:58:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.377+0000 7fa3ff800640 1 -- start start 2024-03-28T17:58:51.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.379+0000 7fa3ff800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3f8161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.379+0000 7fa3ff800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa3f8071b90 0x7fa3f8161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.379+0000 7fa3ff800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80b3470 con 0x7fa3f8072ad0 2024-03-28T17:58:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.379+0000 7fa3ff800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f81629b0 con 0x7fa3f8071b90 2024-03-28T17:58:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.379+0000 7fa3ff800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80b25b0 con 0x7fa3f815e030 2024-03-28T17:58:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3f7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa3f8072ad0 0x7fa3f80ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58356/0 (socket says 172.21.15.62:58356) 2024-03-28T17:58:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3f7e00640 1 -- 172.21.15.62:0/2442493038 learned_addr learned my addr 172.21.15.62:0/2442493038 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3fce00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3f8161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3fd800640 1 --2- 172.21.15.62:0/2442493038 >> v2:172.21.15.81:3300/0 conn(0x7fa3f8071b90 0x7fa3f8161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981314686 0 0) 0x7fa3f80b3470 con 0x7fa3f8072ad0 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.380+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3dc003520 con 0x7fa3f8072ad0 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969878297 0 0) 0x7fa3dc003520 con 0x7fa3f8072ad0 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 msgr2=0x7fa3f8161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3f8161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 >> v2:172.21.15.81:3300/0 conn(0x7fa3f8071b90 msgr2=0x7fa3f8161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 --2- 172.21.15.62:0/2442493038 >> v2:172.21.15.81:3300/0 conn(0x7fa3f8071b90 0x7fa3f8161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f81626d0 con 0x7fa3f8072ad0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fa3e4001670 con 0x7fa3f8072ad0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 >> v1:172.21.15.62:6789/0 conn(0x7fa3f8072ad0 legacy=0x7fa3f80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81629b0 con 0x7fa3dc0039b0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00b9c0 con 0x7fa3dc004800 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00bb40 con 0x7fa3f815e030 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3f7e00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 -- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 msgr2=0x7fa3dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fd800640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3f815e030 0x7fa3dc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 -- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc004800 msgr2=0x7fa3dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:51.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.381+0000 7fa3fce00640 1 -- 172.21.15.62:0/2442493038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc003520 con 0x7fa3dc0039b0 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3fce00640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009ac0 tx=0x7fa3dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cc756c5933e09b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8039070 con 0x7fa3dc0039b0 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e802c6b0 con 0x7fa3dc0039b0 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3f7400640 1 -- 172.21.15.62:0/2442493038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8009690 con 0x7fa3dc0039b0 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3ff800640 1 -- 172.21.15.62:0/2442493038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc0039b0 msgr2=0x7fa3dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3ff800640 1 --2- 172.21.15.62:0/2442493038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009ac0 tx=0x7fa3dc005830 comp rx=0 tx=0).stop 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3fd800640 1 -- 172.21.15.62:0/2442493038 reap_dead start 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3ff800640 1 -- 172.21.15.62:0/2442493038 shutdown_connections 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.383+0000 7fa3ff800640 1 -- 172.21.15.62:0/2442493038 >> 172.21.15.62:0/2442493038 conn(0x7fa3f80b1650 msgr2=0x7fa3f80b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.384+0000 7fa3ff800640 1 -- 172.21.15.62:0/2442493038 shutdown_connections 2024-03-28T17:58:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.384+0000 7fa3ff800640 1 -- 172.21.15.62:0/2442493038 wait complete. 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.385+0000 7fa3ff800640 1 Processor -- start 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.385+0000 7fa3ff800640 1 -- start start 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3dc0039b0 0x7fa3f8155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3f814f650 0x7fa3f8145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80b37a0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8162480 con 0x7fa3f814f650 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.386+0000 7fa3ff800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80723f0 con 0x7fa3dc0039b0 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52572/0 (socket says 172.21.15.62:52572) 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 -- 172.21.15.62:0/1391308015 learned_addr learned my addr 172.21.15.62:0/1391308015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3f7e00640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3f814f650 0x7fa3f8145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fce00640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3dc0039b0 0x7fa3f8155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 -- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3dc0039b0 msgr2=0x7fa3f8155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3dc0039b0 0x7fa3f8155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 -- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3f814f650 msgr2=0x7fa3f8145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3f814f650 0x7fa3f8145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 -- 172.21.15.62:0/1391308015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8153070 con 0x7fa3f81f3d30 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.387+0000 7fa3fd800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80d09f0 tx=0x7fa3f000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.388+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001c070 con 0x7fa3f81f3d30 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.388+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0017400 con 0x7fa3f81f3d30 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.388+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00177f0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.388+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f815fe90 con 0x7fa3f81f3d30 2024-03-28T17:58:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.388+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f8152cb0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.389+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80680c0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.390+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa3f001f070 con 0x7fa3f81f3d30 2024-03-28T17:58:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.391+0000 7fa3f4c00640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 0x7fa3d806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.391+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3f008a6e0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.391+0000 7fa3fce00640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 0x7fa3d806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.392+0000 7fa3fce00640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 0x7fa3d806e0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8036000 tx=0x7fa3e8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:51.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.396+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa3f0012070 con 0x7fa3f81f3d30 2024-03-28T17:58:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.606+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa3f8152a70 con 0x7fa3f81f3d30 2024-03-28T17:58:51.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.951+0000 7fa3f4c00640 1 -- 172.21.15.62:0/1391308015 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa3f0058ec0 con 0x7fa3f81f3d30 2024-03-28T17:58:51.952 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-03-28T17:58:51.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 msgr2=0x7fa3d806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 0x7fa3d806e0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8036000 tx=0x7fa3e8030000 comp rx=0 tx=0).stop 2024-03-28T17:58:51.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 msgr2=0x7fa3f8151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:51.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80d09f0 tx=0x7fa3f000f4a0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 shutdown_connections 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa3d806bc00 0x7fa3d806e0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3f81f3d30 0x7fa3f8151000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3f814f650 0x7fa3f8145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 --2- 172.21.15.62:0/1391308015 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa3dc0039b0 0x7fa3f8155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 >> 172.21.15.62:0/1391308015 conn(0x7fa3f80b1650 msgr2=0x7fa3f80af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 shutdown_connections 2024-03-28T17:58:51.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:51.955+0000 7fa3ff800640 1 -- 172.21.15.62:0/1391308015 wait complete. 2024-03-28T17:58:51.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T17:58:52.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.194+0000 7faf9b200640 1 Processor -- start 2024-03-28T17:58:52.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.195+0000 7faf9b200640 1 -- start start 2024-03-28T17:58:52.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.196+0000 7faf9b200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faf9c0d3b40 0x7faf9c0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.196+0000 7faf9b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf9c0b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.196+0000 7faf9b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0b9b60 con 0x7faf9c0d3f20 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.196+0000 7faf9b200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0d2330 con 0x7faf9c0d3b40 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.196+0000 7faf9b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0d2610 con 0x7faf9c0d33d0 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf9a800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faf9c0d3f20 0x7faf9c0b6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58370/0 (socket says 172.21.15.62:58370) 2024-03-28T17:58:52.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf9a800640 1 -- 172.21.15.62:0/3743691038 learned_addr learned my addr 172.21.15.62:0/3743691038 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf99e00640 1 --2- 172.21.15.62:0/3743691038 >> v2:172.21.15.81:3300/0 conn(0x7faf9c0d3b40 0x7faf9c0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf99400640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf9c0b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502471307 0 0) 0x7faf9c0b9b60 con 0x7faf9c0d3f20 2024-03-28T17:58:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.197+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf80003520 con 0x7faf9c0d3f20 2024-03-28T17:58:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680681368 0 0) 0x7faf80003520 con 0x7faf9c0d3f20 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 msgr2=0x7faf9c0b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf9c0b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 >> v2:172.21.15.81:3300/0 conn(0x7faf9c0d3b40 msgr2=0x7faf9c0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 --2- 172.21.15.62:0/3743691038 >> v2:172.21.15.81:3300/0 conn(0x7faf9c0d3b40 0x7faf9c0d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0d28f0 con 0x7faf9c0d3f20 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7faf940019f0 con 0x7faf9c0d3f20 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 >> v1:172.21.15.62:6789/0 conn(0x7faf9c0d3f20 legacy=0x7faf9c0b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf80003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf80003fa0 0x7faf80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf80004d40 0x7faf800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0d2330 con 0x7faf80004d40 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0d2610 con 0x7faf80003fa0 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8000b9f0 con 0x7faf9c0d33d0 2024-03-28T17:58:52.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf9a800640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf80004d40 0x7faf800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf99e00640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf80003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.198+0000 7faf99400640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf80003fa0 0x7faf80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.199+0000 7faf9a800640 1 -- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 msgr2=0x7faf80003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.199+0000 7faf9a800640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0d33d0 0x7faf80003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.199+0000 7faf9a800640 1 -- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf80003fa0 msgr2=0x7faf80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.199+0000 7faf9a800640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf80003fa0 0x7faf80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.199+0000 7faf9a800640 1 -- 172.21.15.62:0/3743691038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80003520 con 0x7faf80004d40 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.200+0000 7faf9a800640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf80004d40 0x7faf800051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf9400a8a0 tx=0x7faf80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0188c6df721f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.200+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9400a4c0 con 0x7faf80004d40 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.200+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94034650 con 0x7faf80004d40 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.200+0000 7faf98a00640 1 -- 172.21.15.62:0/3743691038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94034a80 con 0x7faf80004d40 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 -- 172.21.15.62:0/3743691038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf80004d40 msgr2=0x7faf800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 --2- 172.21.15.62:0/3743691038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf80004d40 0x7faf800051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf9400a8a0 tx=0x7faf80005830 comp rx=0 tx=0).stop 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9a800640 1 -- 172.21.15.62:0/3743691038 reap_dead start 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 -- 172.21.15.62:0/3743691038 shutdown_connections 2024-03-28T17:58:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 -- 172.21.15.62:0/3743691038 >> 172.21.15.62:0/3743691038 conn(0x7faf9c071ae0 msgr2=0x7faf9c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:52.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 -- 172.21.15.62:0/3743691038 shutdown_connections 2024-03-28T17:58:52.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.201+0000 7faf9b200640 1 -- 172.21.15.62:0/3743691038 wait complete. 2024-03-28T17:58:52.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.203+0000 7faf9b200640 1 Processor -- start 2024-03-28T17:58:52.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.203+0000 7faf9b200640 1 -- start start 2024-03-28T17:58:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.204+0000 7faf9b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.204+0000 7faf9b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf9c154980 0x7faf9c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.204+0000 7faf9b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0cc940 0x7faf9c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf9b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0d3830 con 0x7faf9c1545a0 2024-03-28T17:58:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf9b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0d2eb0 con 0x7faf9c154980 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf9b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15e0f0 con 0x7faf9c0cc940 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52584/0 (socket says 172.21.15.62:52584) 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 -- 172.21.15.62:0/3650362319 learned_addr learned my addr 172.21.15.62:0/3650362319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf9a800640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0cc940 0x7faf9c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99400640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf9c154980 0x7faf9c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 -- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0cc940 msgr2=0x7faf9c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0cc940 0x7faf9c0ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 -- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf9c154980 msgr2=0x7faf9c0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf9c154980 0x7faf9c0c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.205+0000 7faf99e00640 1 -- 172.21.15.62:0/3650362319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0c83a0 con 0x7faf9c1545a0 2024-03-28T17:58:52.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf99e00640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf8c011a70 tx=0x7faf8c011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:52.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00fb50 con 0x7faf9c1545a0 2024-03-28T17:58:52.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00fcf0 con 0x7faf9c1545a0 2024-03-28T17:58:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0c8680 con 0x7faf9c1545a0 2024-03-28T17:58:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00fea0 con 0x7faf9c1545a0 2024-03-28T17:58:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.206+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c0c8c40 con 0x7faf9c1545a0 2024-03-28T17:58:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.207+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c04f400 con 0x7faf9c1545a0 2024-03-28T17:58:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.208+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf8c012040 con 0x7faf9c1545a0 2024-03-28T17:58:52.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.210+0000 7faf92000640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 0x7faf7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:52.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.210+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7faf8c08b4f0 con 0x7faf9c1545a0 2024-03-28T17:58:52.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.211+0000 7faf99400640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 0x7faf7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:52.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.212+0000 7faf99400640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 0x7faf7c06e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf880098d0 tx=0x7faf88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:52.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.215+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faf8c02b020 con 0x7faf9c1545a0 2024-03-28T17:58:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.416+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faf9c158ec0 con 0x7faf9c1545a0 2024-03-28T17:58:52.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.966+0000 7faf92000640 1 -- 172.21.15.62:0/3650362319 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7faf8c019030 con 0x7faf9c1545a0 2024-03-28T17:58:52.966 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-03-28T17:58:52.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.970+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 msgr2=0x7faf7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.970+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 0x7faf7c06e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf880098d0 tx=0x7faf88008040 comp rx=0 tx=0).stop 2024-03-28T17:58:52.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.970+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 msgr2=0x7faf9c0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:52.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.970+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf8c011a70 tx=0x7faf8c011f30 comp rx=0 tx=0).stop 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.970+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 shutdown_connections 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf7c06bbd0 0x7faf7c06e090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf9c0cc940 0x7faf9c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf9c154980 0x7faf9c0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 --2- 172.21.15.62:0/3650362319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf9c1545a0 0x7faf9c0c9d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 >> 172.21.15.62:0/3650362319 conn(0x7faf9c071ae0 msgr2=0x7faf9c0ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:52.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 shutdown_connections 2024-03-28T17:58:52.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:52.971+0000 7faf9b200640 1 -- 172.21.15.62:0/3650362319 wait complete. 2024-03-28T17:58:53.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T17:58:53.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.193+0000 7f3d3f200640 1 Processor -- start 2024-03-28T17:58:53.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.193+0000 7f3d3f200640 1 -- start start 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d380ce780 0x7f3d380d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d380b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3f200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d380d27b0 con 0x7f3d3815e210 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d380d2a90 con 0x7f3d380ce780 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380d2d70 con 0x7f3d380d0b80 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d37400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d380ce780 0x7f3d380d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d37400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d380ce780 0x7f3d380d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52586/0 (socket says 172.21.15.62:52586) 2024-03-28T17:58:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d37400640 1 -- 172.21.15.62:0/2832825467 learned_addr learned my addr 172.21.15.62:0/2832825467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:53.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d37400640 1 -- 172.21.15.62:0/2832825467 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 msgr2=0x7f3d380b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:53.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.195+0000 7f3d3d200640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d380b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.196+0000 7f3d37400640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d380b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.196+0000 7f3d37400640 1 -- 172.21.15.62:0/2832825467 >> v1:172.21.15.81:6789/0 conn(0x7f3d3815e210 legacy=0x7f3d380d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:53.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.196+0000 7f3d37400640 1 -- 172.21.15.62:0/2832825467 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380d3050 con 0x7f3d380ce780 2024-03-28T17:58:53.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.196+0000 7f3d3d200640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d380b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.197+0000 7f3d37400640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d380d0640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009e50 tx=0x7f3d2002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d1765f8a21cb55 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.197+0000 7f3d36a00640 1 -- 172.21.15.62:0/2832825467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2002ca80 con 0x7f3d380ce780 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.197+0000 7f3d36a00640 1 -- 172.21.15.62:0/2832825467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2002cc20 con 0x7f3d380ce780 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.197+0000 7f3d36a00640 1 -- 172.21.15.62:0/2832825467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200343f0 con 0x7f3d380ce780 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 -- 172.21.15.62:0/2832825467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 msgr2=0x7f3d380d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d380d0640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009e50 tx=0x7f3d2002bee0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 -- 172.21.15.62:0/2832825467 shutdown_connections 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d380b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 --2- 172.21.15.62:0/2832825467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d380d0640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 -- 172.21.15.62:0/2832825467 >> 172.21.15.62:0/2832825467 conn(0x7f3d38066dd0 msgr2=0x7f3d380671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:53.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 -- 172.21.15.62:0/2832825467 shutdown_connections 2024-03-28T17:58:53.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.198+0000 7f3d3f200640 1 -- 172.21.15.62:0/2832825467 wait complete. 2024-03-28T17:58:53.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.199+0000 7f3d3f200640 1 Processor -- start 2024-03-28T17:58:53.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.200+0000 7f3d3f200640 1 -- start start 2024-03-28T17:58:53.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.200+0000 7f3d3f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d381f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3815e210 0x7f3d380b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3815e5f0 con 0x7f3d380ce780 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380d1010 con 0x7f3d3815e210 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380b7620 con 0x7f3d380d0b80 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52598/0 (socket says 172.21.15.62:52598) 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 -- 172.21.15.62:0/340971547 learned_addr learned my addr 172.21.15.62:0/340971547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37400640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d381f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d3d200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3815e210 0x7f3d380b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 -- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 msgr2=0x7f3d381f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d381f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 -- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3815e210 msgr2=0x7f3d380b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3815e210 0x7f3d380b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:53.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.201+0000 7f3d37e00640 1 -- 172.21.15.62:0/340971547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d381f4560 con 0x7f3d380ce780 2024-03-28T17:58:53.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d37e00640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b3f0 tx=0x7f3d2800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:53.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28015020 con 0x7f3d380ce780 2024-03-28T17:58:53.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28011830 con 0x7f3d380ce780 2024-03-28T17:58:53.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d381f4840 con 0x7f3d380ce780 2024-03-28T17:58:53.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28011c20 con 0x7f3d380ce780 2024-03-28T17:58:53.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.202+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381f4b20 con 0x7f3d380ce780 2024-03-28T17:58:53.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.203+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d380680c0 con 0x7f3d380ce780 2024-03-28T17:58:53.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.204+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3d28018020 con 0x7f3d380ce780 2024-03-28T17:58:53.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.205+0000 7f3d2fe00640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 0x7f3d1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:53.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.205+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3d2808a740 con 0x7f3d380ce780 2024-03-28T17:58:53.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.206+0000 7f3d37400640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 0x7f3d1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:53.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.207+0000 7f3d37400640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 0x7f3d1406e090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d20035000 tx=0x7f3d20030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:53.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.210+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3d2801f030 con 0x7f3d380ce780 2024-03-28T17:58:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.449+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3d381f5390 con 0x7f3d380ce780 2024-03-28T17:58:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.988+0000 7f3d2fe00640 1 -- 172.21.15.62:0/340971547 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3d381f5390 con 0x7f3d380ce780 2024-03-28T17:58:53.992 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-03-28T17:58:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.997+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 msgr2=0x7f3d1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.997+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 0x7f3d1406e090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d20035000 tx=0x7f3d20030000 comp rx=0 tx=0).stop 2024-03-28T17:58:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.997+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 msgr2=0x7f3d38159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:53.997+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b3f0 tx=0x7f3d2800b8b0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 shutdown_connections 2024-03-28T17:58:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3d1406bbd0 0x7f3d1406e090 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3815e210 0x7f3d380b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3d380d0b80 0x7f3d381f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 --2- 172.21.15.62:0/340971547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d380ce780 0x7f3d38159c40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 >> 172.21.15.62:0/340971547 conn(0x7f3d38066dd0 msgr2=0x7f3d3815feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 shutdown_connections 2024-03-28T17:58:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.000+0000 7f3d3f200640 1 -- 172.21.15.62:0/340971547 wait complete. 2024-03-28T17:58:54.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.212+0000 7f4f66200640 1 Processor -- start 2024-03-28T17:58:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.212+0000 7f4f66200640 1 -- start start 2024-03-28T17:58:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.213+0000 7f4f66200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f6015df90 0x7f4f60161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f66200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f60162200 con 0x7f4f600cf280 2024-03-28T17:58:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f66200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f601624e0 con 0x7f4f600b98e0 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f66200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f600d2110 con 0x7f4f6015df90 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f5f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f6015df90 0x7f4f60161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f5fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4f600cf280 0x7f4f600d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56544/0 (socket says 172.21.15.62:56544) 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.214+0000 7f4f5fe00640 1 -- 172.21.15.62:0/1915376057 learned_addr learned my addr 172.21.15.62:0/1915376057 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.215+0000 7f4f5f400640 1 -- 172.21.15.62:0/1915376057 >> v1:172.21.15.81:6789/0 conn(0x7f4f600b98e0 legacy=0x7f4f600ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.215+0000 7f4f5ea00640 1 -- 172.21.15.62:0/1915376057 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028730347 0 0) 0x7f4f60162200 con 0x7f4f600cf280 2024-03-28T17:58:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.215+0000 7f4f5f400640 1 -- 172.21.15.62:0/1915376057 >> v1:172.21.15.62:6789/0 conn(0x7f4f600cf280 legacy=0x7f4f600d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.215+0000 7f4f5f400640 1 -- 172.21.15.62:0/1915376057 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f601627c0 con 0x7f4f6015df90 2024-03-28T17:58:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.217+0000 7f4f5f400640 1 --2- 172.21.15.62:0/1915376057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6015df90 0x7f4f60161b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb117f58673cdc57 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.217+0000 7f4f5ea00640 1 -- 172.21.15.62:0/1915376057 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50035020 con 0x7f4f6015df90 2024-03-28T17:58:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.217+0000 7f4f5ea00640 1 -- 172.21.15.62:0/1915376057 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f50031830 con 0x7f4f6015df90 2024-03-28T17:58:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.217+0000 7f4f5ea00640 1 -- 172.21.15.62:0/1915376057 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50031c00 con 0x7f4f6015df90 2024-03-28T17:58:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 -- 172.21.15.62:0/1915376057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6015df90 msgr2=0x7f4f60161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 --2- 172.21.15.62:0/1915376057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6015df90 0x7f4f60161b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002b8d0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 -- 172.21.15.62:0/1915376057 shutdown_connections 2024-03-28T17:58:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 --2- 172.21.15.62:0/1915376057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6015df90 0x7f4f60161b70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 -- 172.21.15.62:0/1915376057 >> 172.21.15.62:0/1915376057 conn(0x7f4f60066fb0 msgr2=0x7f4f600673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:54.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 -- 172.21.15.62:0/1915376057 shutdown_connections 2024-03-28T17:58:54.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.218+0000 7f4f66200640 1 -- 172.21.15.62:0/1915376057 wait complete. 2024-03-28T17:58:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.220+0000 7f4f66200640 1 Processor -- start 2024-03-28T17:58:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.220+0000 7f4f66200640 1 -- start start 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f600b98e0 0x7f4f6015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 0x7f4f601480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015e370 con 0x7f4f600cf280 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f60162aa0 con 0x7f4f600b98e0 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f66200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015e4f0 con 0x7f4f6015df90 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46682/0 (socket says 172.21.15.62:46682) 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 -- 172.21.15.62:0/253583696 learned_addr learned my addr 172.21.15.62:0/253583696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:54.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f5f400640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f600b98e0 0x7f4f6015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f5fe00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 0x7f4f601480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 -- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 msgr2=0x7f4f601480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 0x7f4f601480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 -- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f600b98e0 msgr2=0x7f4f6015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f600b98e0 0x7f4f6015af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.221+0000 7f4f56a00640 1 -- 172.21.15.62:0/253583696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600ca7b0 con 0x7f4f600cf280 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f5fe00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 0x7f4f601480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f56a00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c450 tx=0x7f4f4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f480052d0 con 0x7f4f600cf280 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48005470 con 0x7f4f600cf280 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f60159e90 con 0x7f4f600cf280 2024-03-28T17:58:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48012870 con 0x7f4f600cf280 2024-03-28T17:58:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.222+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6015a450 con 0x7f4f600cf280 2024-03-28T17:58:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.223+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f600680c0 con 0x7f4f600cf280 2024-03-28T17:58:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.224+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4f48012a10 con 0x7f4f600cf280 2024-03-28T17:58:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.225+0000 7f4f57e00640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 0x7f4f3c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.225+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f4808a740 con 0x7f4f600cf280 2024-03-28T17:58:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.226+0000 7f4f5f400640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 0x7f4f3c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.227+0000 7f4f5f400640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 0x7f4f3c06e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f50036000 tx=0x7f4f5003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.230+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4f48059190 con 0x7f4f600cf280 2024-03-28T17:58:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.433+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f600d33c0 con 0x7f4f600cf280 2024-03-28T17:58:54.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.436+0000 7f4f57e00640 1 -- 172.21.15.62:0/253583696 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f4f48058a90 con 0x7f4f600cf280 2024-03-28T17:58:54.437 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:54.437 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:53.451170+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 msgr2=0x7f4f3c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 0x7f4f3c06e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f50036000 tx=0x7f4f5003c040 comp rx=0 tx=0).stop 2024-03-28T17:58:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 msgr2=0x7f4f600ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c450 tx=0x7f4f4800c910 comp rx=0 tx=0).stop 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 shutdown_connections 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4f3c06bbd0 0x7f4f3c06e090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4f6015df90 0x7f4f601480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f600cf280 0x7f4f600ca270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 --2- 172.21.15.62:0/253583696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f600b98e0 0x7f4f6015af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 >> 172.21.15.62:0/253583696 conn(0x7f4f60066fb0 msgr2=0x7f4f600b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 shutdown_connections 2024-03-28T17:58:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.439+0000 7f4f66200640 1 -- 172.21.15.62:0/253583696 wait complete. 2024-03-28T17:58:54.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.681+0000 7ff8eee00640 1 Processor -- start 2024-03-28T17:58:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.681+0000 7ff8eee00640 1 -- start start 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.682+0000 7ff8eee00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff8e80722b0 0x7ff8e8072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.682+0000 7ff8eee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e8071ed0 con 0x7ff8e806caa0 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8eee00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e8070a40 con 0x7ff8e815b9c0 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8eee00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e8071330 con 0x7ff8e80722b0 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8ee400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff8e806caa0 0x7ff8e806cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56546/0 (socket says 172.21.15.62:56546) 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8ee400640 1 -- 172.21.15.62:0/4280978420 learned_addr learned my addr 172.21.15.62:0/4280978420 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8ed000640 1 --2- 172.21.15.62:0/4280978420 >> v2:172.21.15.81:3300/0 conn(0x7ff8e80722b0 0x7ff8e8072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946369557 0 0) 0x7ff8e8071ed0 con 0x7ff8e806caa0 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.683+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0003520 con 0x7ff8e806caa0 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248753671 0 0) 0x7ff8d0003520 con 0x7ff8e806caa0 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 >> v2:172.21.15.81:3300/0 conn(0x7ff8e80722b0 msgr2=0x7ff8e8072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 --2- 172.21.15.62:0/4280978420 >> v2:172.21.15.81:3300/0 conn(0x7ff8e80722b0 0x7ff8e8072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 >> v1:172.21.15.81:6789/0 conn(0x7ff8e815b9c0 legacy=0x7ff8e80717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e80b3af0 con 0x7ff8e806caa0 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7ff8e00019f0 con 0x7ff8e806caa0 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 >> v1:172.21.15.62:6789/0 conn(0x7ff8e806caa0 legacy=0x7ff8e806cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d00039b0 0x7ff8d0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8071330 con 0x7ff8d0004800 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000c140 con 0x7ff8d0005210 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000c340 con 0x7ff8d00039b0 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8eda00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d00039b0 0x7ff8d0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ee400640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 -- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d00039b0 msgr2=0x7ff8d0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d00039b0 0x7ff8d0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 -- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8d0005210 msgr2=0x7ff8d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ed000640 1 -- 172.21.15.62:0/4280978420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0003520 con 0x7ff8d0004800 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ee400640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.684+0000 7ff8ee400640 1 -- 172.21.15.62:0/4280978420 reap_dead start 2024-03-28T17:58:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.685+0000 7ff8ed000640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009ab0 tx=0x7ff8d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee712cda56ae8d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8009680 con 0x7ff8d0004800 2024-03-28T17:58:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8005990 con 0x7ff8d0004800 2024-03-28T17:58:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8e7e00640 1 -- 172.21.15.62:0/4280978420 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8005d80 con 0x7ff8d0004800 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8eee00640 1 -- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 msgr2=0x7ff8d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8eee00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009ab0 tx=0x7ff8d0005830 comp rx=0 tx=0).stop 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8eee00640 1 -- 172.21.15.62:0/4280978420 shutdown_connections 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8eee00640 1 --2- 172.21.15.62:0/4280978420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.686+0000 7ff8eee00640 1 -- 172.21.15.62:0/4280978420 >> 172.21.15.62:0/4280978420 conn(0x7ff8e80b1730 msgr2=0x7ff8e80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.687+0000 7ff8eee00640 1 -- 172.21.15.62:0/4280978420 shutdown_connections 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.687+0000 7ff8eee00640 1 -- 172.21.15.62:0/4280978420 wait complete. 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.688+0000 7ff8eee00640 1 Processor -- start 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.688+0000 7ff8eee00640 1 -- start start 2024-03-28T17:58:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d0004800 0x7ff8e815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 0x7ff8e815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8070fc0 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8070ce0 con 0x7ff8e814c820 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.689+0000 7ff8eee00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e815bda0 con 0x7ff8d0004800 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46702/0 (socket says 172.21.15.62:46702) 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 -- 172.21.15.62:0/2238592216 learned_addr learned my addr 172.21.15.62:0/2238592216 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8eda00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d0004800 0x7ff8e815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ed000640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 0x7ff8e815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 -- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d0004800 msgr2=0x7ff8e815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d0004800 0x7ff8e815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 -- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 msgr2=0x7ff8e815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 0x7ff8e815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 -- 172.21.15.62:0/2238592216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e81f5330 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ed000640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 0x7ff8e815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8ee400640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001580 tx=0x7ff8e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.690+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0004d60 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.691+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000bb80 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.691+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e80ce410 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.691+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0007070 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.691+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8155640 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.692+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e80b4590 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.693+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff8e001b020 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.694+0000 7ff8e5600640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 0x7ff8cc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.694+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e008ac00 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.694+0000 7ff8eda00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 0x7ff8cc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.695+0000 7ff8eda00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 0x7ff8cc06e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8157960 tx=0x7ff8dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:54.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.699+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff8e80b4590 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.903+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8e8073490 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.904+0000 7ff8e5600640 1 -- 172.21.15.62:0/2238592216 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16570 (secure 0 0 0) 0x7ff8e81f5330 con 0x7ff8e80cd6a0 2024-03-28T17:58:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:54.905 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:53.451170+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.907+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 msgr2=0x7ff8cc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.907+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 0x7ff8cc06e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8157960 tx=0x7ff8dc009040 comp rx=0 tx=0).stop 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.907+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 msgr2=0x7ff8e80cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.907+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001580 tx=0x7ff8e000b4a0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.907+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 shutdown_connections 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff8cc06bbd0 0x7ff8cc06e090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8e80cd6a0 0x7ff8e80cdb20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e814c820 0x7ff8e815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 --2- 172.21.15.62:0/2238592216 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff8d0004800 0x7ff8e815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 >> 172.21.15.62:0/2238592216 conn(0x7ff8e80b1730 msgr2=0x7ff8e8072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 shutdown_connections 2024-03-28T17:58:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:54.908+0000 7ff8eee00640 1 -- 172.21.15.62:0/2238592216 wait complete. 2024-03-28T17:58:54.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T17:58:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.159+0000 7f2b60200640 1 Processor -- start 2024-03-28T17:58:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.159+0000 7f2b60200640 1 -- start start 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b60200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b58072690 0x7f2b580cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b60200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2b58072ad0 0x7f2b580ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b60200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5815d1b0 con 0x7f2b580ae720 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b60200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b58070d50 con 0x7f2b58072ad0 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b60200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58071030 con 0x7f2b58072690 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b5e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2b58072ad0 0x7f2b580ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b5e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2b58072ad0 0x7f2b580ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46714/0 (socket says 172.21.15.62:46714) 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b5e200640 1 -- 172.21.15.62:0/4173923768 learned_addr learned my addr 172.21.15.62:0/4173923768 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.161+0000 7f2b5ce00640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b58072690 0x7f2b580cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.162+0000 7f2b5e200640 1 -- 172.21.15.62:0/4173923768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b58072690 msgr2=0x7f2b580cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.162+0000 7f2b5e200640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b58072690 0x7f2b580cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.162+0000 7f2b5e200640 1 -- 172.21.15.62:0/4173923768 >> v1:172.21.15.81:6789/0 conn(0x7f2b580ae720 legacy=0x7f2b580aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.162+0000 7f2b5e200640 1 -- 172.21.15.62:0/4173923768 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58071310 con 0x7f2b58072ad0 2024-03-28T17:58:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.163+0000 7f2b5e200640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072ad0 0x7f2b580ce390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009e50 tx=0x7f2b5002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80aeff363f30b726 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.163+0000 7f2b57e00640 1 -- 172.21.15.62:0/4173923768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5002caf0 con 0x7f2b58072ad0 2024-03-28T17:58:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.163+0000 7f2b57e00640 1 -- 172.21.15.62:0/4173923768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5002cc90 con 0x7f2b58072ad0 2024-03-28T17:58:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.163+0000 7f2b57e00640 1 -- 172.21.15.62:0/4173923768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50034610 con 0x7f2b58072ad0 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 -- 172.21.15.62:0/4173923768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072ad0 msgr2=0x7f2b580ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072ad0 0x7f2b580ce390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009e50 tx=0x7f2b5002bf50 comp rx=0 tx=0).stop 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 -- 172.21.15.62:0/4173923768 shutdown_connections 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072ad0 0x7f2b580ce390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 --2- 172.21.15.62:0/4173923768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b58072690 0x7f2b580cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 -- 172.21.15.62:0/4173923768 >> 172.21.15.62:0/4173923768 conn(0x7f2b580b1650 msgr2=0x7f2b580b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:55.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 -- 172.21.15.62:0/4173923768 shutdown_connections 2024-03-28T17:58:55.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.164+0000 7f2b60200640 1 -- 172.21.15.62:0/4173923768 wait complete. 2024-03-28T17:58:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.166+0000 7f2b60200640 1 Processor -- start 2024-03-28T17:58:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.166+0000 7f2b60200640 1 -- start start 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b58072ad0 0x7f2b581f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 0x7f2b5814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58070a40 con 0x7f2b58072690 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580718d0 con 0x7f2b58072ad0 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b60200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58153030 con 0x7f2b580ae720 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b5d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b5d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46718/0 (socket says 172.21.15.62:46718) 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b5d800640 1 -- 172.21.15.62:0/960917251 learned_addr learned my addr 172.21.15.62:0/960917251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b5e200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 0x7f2b5814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.167+0000 7f2b5ce00640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b58072ad0 0x7f2b581f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 -- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 msgr2=0x7f2b5814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 0x7f2b5814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 -- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b58072ad0 msgr2=0x7f2b581f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b58072ad0 0x7f2b581f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 -- 172.21.15.62:0/960917251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580aa940 con 0x7f2b58072690 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5e200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 0x7f2b5814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b5d800640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c3c0 tx=0x7f2b4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0052d0 con 0x7f2b58072690 2024-03-28T17:58:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c005490 con 0x7f2b58072690 2024-03-28T17:58:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c007260 con 0x7f2b58072690 2024-03-28T17:58:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.168+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b580aac20 con 0x7f2b58072690 2024-03-28T17:58:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.169+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b581f5b50 con 0x7f2b58072690 2024-03-28T17:58:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.170+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58071df0 con 0x7f2b58072690 2024-03-28T17:58:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.170+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2b4c007400 con 0x7f2b58072690 2024-03-28T17:58:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.171+0000 7f2b55600640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 0x7f2b3806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.171+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b4c089ec0 con 0x7f2b58072690 2024-03-28T17:58:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.172+0000 7f2b5ce00640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 0x7f2b3806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.173+0000 7f2b5ce00640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 0x7f2b3806e0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b5806e1b0 tx=0x7f2b44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.176+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2b58071df0 con 0x7f2b58072690 2024-03-28T17:58:55.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:55.416+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2b58160620 con 0x7f2b58072690 2024-03-28T17:58:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.012+0000 7f2b55600640 1 -- 172.21.15.62:0/960917251 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2b58160620 con 0x7f2b58072690 2024-03-28T17:58:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 msgr2=0x7f2b3806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 0x7f2b3806e0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b5806e1b0 tx=0x7f2b44008040 comp rx=0 tx=0).stop 2024-03-28T17:58:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 msgr2=0x7f2b580ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c3c0 tx=0x7f2b4c00c880 comp rx=0 tx=0).stop 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 shutdown_connections 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2b3806bc00 0x7f2b3806e0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2b580ae720 0x7f2b5814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b58072ad0 0x7f2b581f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 --2- 172.21.15.62:0/960917251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b58072690 0x7f2b580ae0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 >> 172.21.15.62:0/960917251 conn(0x7f2b580b1650 msgr2=0x7f2b58152270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.016+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 shutdown_connections 2024-03-28T17:58:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.017+0000 7f2b60200640 1 -- 172.21.15.62:0/960917251 wait complete. 2024-03-28T17:58:56.037 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T17:58:56.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T17:58:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.259+0000 7fa0ca200640 1 Processor -- start 2024-03-28T17:58:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.259+0000 7fa0ca200640 1 -- start start 2024-03-28T17:58:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0ca200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0c40ae720 0x7fa0c40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c415f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0ca200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c415d1b0 con 0x7fa0c4072ad0 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0ca200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40af040 con 0x7fa0c40ae720 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0ca200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4070d50 con 0x7fa0c4072690 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0c3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0c40ae720 0x7fa0c40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.261+0000 7fa0c3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0c40ae720 0x7fa0c40aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46734/0 (socket says 172.21.15.62:46734) 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c3400640 1 -- 172.21.15.62:0/951099410 learned_addr learned my addr 172.21.15.62:0/951099410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c2a00640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c3400640 1 -- 172.21.15.62:0/951099410 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 msgr2=0x7fa0c415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c3400640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c415f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c3400640 1 -- 172.21.15.62:0/951099410 >> v1:172.21.15.81:6789/0 conn(0x7fa0c4072ad0 legacy=0x7fa0c415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.262+0000 7fa0c3400640 1 -- 172.21.15.62:0/951099410 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4071030 con 0x7fa0c40ae720 2024-03-28T17:58:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.263+0000 7fa0c3400640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c40aeb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d6ce7f5bc66b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.263+0000 7fa0c2000640 1 -- 172.21.15.62:0/951099410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4034070 con 0x7fa0c40ae720 2024-03-28T17:58:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0c2000640 1 -- 172.21.15.62:0/951099410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4030840 con 0x7fa0c40ae720 2024-03-28T17:58:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0c2000640 1 -- 172.21.15.62:0/951099410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4030c10 con 0x7fa0c40ae720 2024-03-28T17:58:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 -- 172.21.15.62:0/951099410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 msgr2=0x7fa0c40aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c40aeb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b402b980 comp rx=0 tx=0).stop 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 -- 172.21.15.62:0/951099410 shutdown_connections 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 --2- 172.21.15.62:0/951099410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c40aeb00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 -- 172.21.15.62:0/951099410 >> 172.21.15.62:0/951099410 conn(0x7fa0c40b1650 msgr2=0x7fa0c40b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 -- 172.21.15.62:0/951099410 shutdown_connections 2024-03-28T17:58:56.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.264+0000 7fa0ca200640 1 -- 172.21.15.62:0/951099410 wait complete. 2024-03-28T17:58:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.266+0000 7fa0ca200640 1 Processor -- start 2024-03-28T17:58:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.266+0000 7fa0ca200640 1 -- start start 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c41e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 0x7fa0c40adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40718d0 con 0x7fa0c40ae720 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40715f0 con 0x7fa0c4072ad0 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0ca200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4070a40 con 0x7fa0c4072690 2024-03-28T17:58:56.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46738/0 (socket says 172.21.15.62:46738) 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.267+0000 7fa0c3e00640 1 -- 172.21.15.62:0/2860680388 learned_addr learned my addr 172.21.15.62:0/2860680388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3400640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c41e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c2a00640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 0x7fa0c40adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 -- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 msgr2=0x7fa0c41e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c41e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 -- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 msgr2=0x7fa0c40adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 0x7fa0c40adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 -- 172.21.15.62:0/2860680388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c41e69c0 con 0x7fa0c40ae720 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c2a00640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 0x7fa0c40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0c3e00640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c730 tx=0x7fa0bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.268+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc003b70 con 0x7fa0c40ae720 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.269+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc003d10 con 0x7fa0c40ae720 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.269+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00aa90 con 0x7fa0c40ae720 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.269+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c41e6ca0 con 0x7fa0c40ae720 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.269+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41e7260 con 0x7fa0c40ae720 2024-03-28T17:58:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.270+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40680c0 con 0x7fa0c40ae720 2024-03-28T17:58:56.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.271+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa0bc00ac30 con 0x7fa0c40ae720 2024-03-28T17:58:56.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.272+0000 7fa0bb400640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 0x7fa0a406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:56.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.272+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0bc08a9b0 con 0x7fa0c40ae720 2024-03-28T17:58:56.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.273+0000 7fa0c3400640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 0x7fa0a406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.274+0000 7fa0c3400640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 0x7fa0a406e0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4036000 tx=0x7fa0b403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.277+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa0bc059490 con 0x7fa0c40ae720 2024-03-28T17:58:56.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:56.520+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa0c40b2420 con 0x7fa0c40ae720 2024-03-28T17:58:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.023+0000 7fa0bb400640 1 -- 172.21.15.62:0/2860680388 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0bc058d90 con 0x7fa0c40ae720 2024-03-28T17:58:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.026+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 msgr2=0x7fa0a406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.026+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 0x7fa0a406e0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4036000 tx=0x7fa0b403b040 comp rx=0 tx=0).stop 2024-03-28T17:58:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.026+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 msgr2=0x7fa0c414e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.026+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c730 tx=0x7fa0bc00cbf0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 shutdown_connections 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0a406bc00 0x7fa0a406e0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0c40ae720 0x7fa0c414e350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0c4072ad0 0x7fa0c40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 --2- 172.21.15.62:0/2860680388 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0c4072690 0x7fa0c41e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 >> 172.21.15.62:0/2860680388 conn(0x7fa0c40b1650 msgr2=0x7fa0c40b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:57.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 shutdown_connections 2024-03-28T17:58:57.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.027+0000 7fa0ca200640 1 -- 172.21.15.62:0/2860680388 wait complete. 2024-03-28T17:58:57.051 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T17:58:57.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T17:58:57.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.278+0000 7f9b0fe00640 1 Processor -- start 2024-03-28T17:58:57.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.278+0000 7f9b0fe00640 1 -- start start 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.279+0000 7f9b0fe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 0x7f9b100b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.279+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.279+0000 7f9b0fe00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100b6e60 con 0x7f9b100b99f0 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0fe00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b100b6c30 con 0x7f9b1015dd60 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0fe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b7100 con 0x7f9b100b5e00 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0ea00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 0x7f9b100b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:53986/0 (socket says 172.21.15.62:53986) 2024-03-28T17:58:57.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.280+0000 7f9b0f400640 1 -- 172.21.15.62:0/737542357 learned_addr learned my addr 172.21.15.62:0/737542357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:57.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0d600640 1 -- 172.21.15.62:0/737542357 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404513901 0 0) 0x7f9b100b6e60 con 0x7f9b100b99f0 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0d600640 1 -- 172.21.15.62:0/737542357 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af4003520 con 0x7f9b100b99f0 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0f400640 1 -- 172.21.15.62:0/737542357 >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 msgr2=0x7f9b100b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0f400640 1 --2- 172.21.15.62:0/737542357 >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 0x7f9b100b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0f400640 1 -- 172.21.15.62:0/737542357 >> v1:172.21.15.81:6789/0 conn(0x7f9b100b99f0 legacy=0x7f9b100b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0f400640 1 -- 172.21.15.62:0/737542357 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100b73e0 con 0x7f9b100b5e00 2024-03-28T17:58:57.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.281+0000 7f9b0ea00640 1 --2- 172.21.15.62:0/737542357 >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 0x7f9b100b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:57.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.282+0000 7f9b0f400640 1 --2- 172.21.15.62:0/737542357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009e50 tx=0x7f9b0802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888053479fb800ba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0d600640 1 -- 172.21.15.62:0/737542357 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0802cb90 con 0x7f9b100b5e00 2024-03-28T17:58:57.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0d600640 1 -- 172.21.15.62:0/737542357 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0802cd30 con 0x7f9b100b5e00 2024-03-28T17:58:57.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0d600640 1 -- 172.21.15.62:0/737542357 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080345f0 con 0x7f9b100b5e00 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 -- 172.21.15.62:0/737542357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 msgr2=0x7f9b100b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/737542357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009e50 tx=0x7f9b0802bff0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 -- 172.21.15.62:0/737542357 shutdown_connections 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/737542357 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b100b5e00 0x7f9b100b7ec0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/737542357 >> v2:172.21.15.81:3300/0 conn(0x7f9b1015dd60 0x7f9b100b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 -- 172.21.15.62:0/737542357 >> 172.21.15.62:0/737542357 conn(0x7f9b10071ae0 msgr2=0x7f9b10071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.283+0000 7f9b0fe00640 1 -- 172.21.15.62:0/737542357 shutdown_connections 2024-03-28T17:58:57.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.284+0000 7f9b0fe00640 1 -- 172.21.15.62:0/737542357 wait complete. 2024-03-28T17:58:57.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.285+0000 7f9b0fe00640 1 Processor -- start 2024-03-28T17:58:57.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.285+0000 7f9b0fe00640 1 -- start start 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b100b99f0 0x7f9b100c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b1015dd60 0x7f9b100c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1015e140 con 0x7f9b100b5e00 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b79a0 con 0x7f9b100b99f0 2024-03-28T17:58:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0fe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b6370 con 0x7f9b1015dd60 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.286+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46752/0 (socket says 172.21.15.62:46752) 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 -- 172.21.15.62:0/1631738150 learned_addr learned my addr 172.21.15.62:0/1631738150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0e000640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b100b99f0 0x7f9b100c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0f400640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b1015dd60 0x7f9b100c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 -- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b1015dd60 msgr2=0x7f9b100c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b1015dd60 0x7f9b100c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 -- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b100b99f0 msgr2=0x7f9b100c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b100b99f0 0x7f9b100c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 -- 172.21.15.62:0/1631738150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100ca590 con 0x7f9b100b5e00 2024-03-28T17:58:57.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.287+0000 7f9b0ea00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e990 tx=0x7f9b0000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.288+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00017070 con 0x7f9b100b5e00 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.288+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000c9e0 con 0x7f9b100b5e00 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.288+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000cdb0 con 0x7f9b100b5e00 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.288+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b101526c0 con 0x7f9b100b5e00 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.288+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b101e78c0 con 0x7f9b100b5e00 2024-03-28T17:58:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.289+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b10072b90 con 0x7f9b100b5e00 2024-03-28T17:58:57.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.290+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9b0001a070 con 0x7f9b100b5e00 2024-03-28T17:58:57.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.291+0000 7f9b06a00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 0x7f9aec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.291+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b0008a920 con 0x7f9b100b5e00 2024-03-28T17:58:57.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.292+0000 7f9b0e000640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 0x7f9aec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.293+0000 7f9b0e000640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 0x7f9aec06e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b100cdf70 tx=0x7f9af8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.296+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9b00059480 con 0x7f9b100b5e00 2024-03-28T17:58:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.538+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b1004f4e0 con 0x7f9b100b5e00 2024-03-28T17:58:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.538+0000 7f9b06a00640 1 -- 172.21.15.62:0/1631738150 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f9b00058d80 con 0x7f9b100b5e00 2024-03-28T17:58:57.539 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:57.539 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T17:58:57.539 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T17:58:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 msgr2=0x7f9aec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 0x7f9aec06e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b100cdf70 tx=0x7f9af8009040 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 msgr2=0x7f9b101516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e990 tx=0x7f9b0000ee50 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 shutdown_connections 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9aec06bbd0 0x7f9aec06e090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9b1015dd60 0x7f9b100c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b100b99f0 0x7f9b100c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 --2- 172.21.15.62:0/1631738150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b100b5e00 0x7f9b101516a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 >> 172.21.15.62:0/1631738150 conn(0x7f9b10071ae0 msgr2=0x7f9b100b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.541+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 shutdown_connections 2024-03-28T17:58:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.542+0000 7f9b0fe00640 1 -- 172.21.15.62:0/1631738150 wait complete. 2024-03-28T17:58:57.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.769+0000 7faf4da00640 1 Processor -- start 2024-03-28T17:58:57.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.770+0000 7faf4da00640 1 -- start start 2024-03-28T17:58:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.771+0000 7faf4da00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf48072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf4da00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48071ed0 con 0x7faf4806caa0 2024-03-28T17:58:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf4da00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48070a40 con 0x7faf4815b9c0 2024-03-28T17:58:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf4da00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf48072c50 con 0x7faf480722b0 2024-03-28T17:58:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf4d000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faf4806caa0 0x7faf4806cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56554/0 (socket says 172.21.15.62:56554) 2024-03-28T17:58:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf4d000640 1 -- 172.21.15.62:0/1202565286 learned_addr learned my addr 172.21.15.62:0/1202565286 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf3fe00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf48072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750462971 0 0) 0x7faf48071ed0 con 0x7faf4806caa0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.772+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf2c003520 con 0x7faf4806caa0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827339076 0 0) 0x7faf2c003520 con 0x7faf4806caa0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 msgr2=0x7faf48072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf48072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 >> v1:172.21.15.81:6789/0 conn(0x7faf4815b9c0 legacy=0x7faf480717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480b3af0 con 0x7faf4806caa0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7faf400019f0 con 0x7faf4806caa0 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 >> v1:172.21.15.62:6789/0 conn(0x7faf4806caa0 legacy=0x7faf4806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf2c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2c003f20 0x7faf2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf2c004d40 0x7faf2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c00ba70 con 0x7faf2c003f20 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c00bbf0 con 0x7faf2c004d40 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c00bdf0 con 0x7faf480722b0 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf3fe00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2c003f20 0x7faf2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf47e00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf2c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.773+0000 7faf4d000640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf2c004d40 0x7faf2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf3fe00640 1 -- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 msgr2=0x7faf2c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf3fe00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf2c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf3fe00640 1 -- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf2c004d40 msgr2=0x7faf2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf3fe00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf2c004d40 0x7faf2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf3fe00640 1 -- 172.21.15.62:0/1202565286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c003520 con 0x7faf2c003f20 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.774+0000 7faf47e00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf480722b0 0x7faf2c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.775+0000 7faf3fe00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2c003f20 0x7faf2c004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ab0 tx=0x7faf2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d69df964e2c932 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.775+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30009680 con 0x7faf2c003f20 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.775+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30005990 con 0x7faf2c003f20 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.775+0000 7faf47400640 1 -- 172.21.15.62:0/1202565286 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30005d80 con 0x7faf2c003f20 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 -- 172.21.15.62:0/1202565286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2c003f20 msgr2=0x7faf2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 --2- 172.21.15.62:0/1202565286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2c003f20 0x7faf2c004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ab0 tx=0x7faf2c005830 comp rx=0 tx=0).stop 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4d000640 1 -- 172.21.15.62:0/1202565286 reap_dead start 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 -- 172.21.15.62:0/1202565286 shutdown_connections 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 -- 172.21.15.62:0/1202565286 >> 172.21.15.62:0/1202565286 conn(0x7faf480b1730 msgr2=0x7faf480b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 -- 172.21.15.62:0/1202565286 shutdown_connections 2024-03-28T17:58:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.776+0000 7faf4da00640 1 -- 172.21.15.62:0/1202565286 wait complete. 2024-03-28T17:58:57.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.778+0000 7faf4da00640 1 Processor -- start 2024-03-28T17:58:57.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.778+0000 7faf4da00640 1 -- start start 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf2c003f20 0x7faf48152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf480cdee0 0x7faf480ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48070ce0 con 0x7faf4814d6d0 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48071330 con 0x7faf480cdee0 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4da00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf48070fc0 con 0x7faf2c003f20 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46770/0 (socket says 172.21.15.62:46770) 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 -- 172.21.15.62:0/2360846954 learned_addr learned my addr 172.21.15.62:0/2360846954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf47e00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf2c003f20 0x7faf48152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf4d000640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf480cdee0 0x7faf480ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 -- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf2c003f20 msgr2=0x7faf48152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf2c003f20 0x7faf48152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 -- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf480cdee0 msgr2=0x7faf480ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf480cdee0 0x7faf480ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.779+0000 7faf3fe00640 1 -- 172.21.15.62:0/2360846954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48153670 con 0x7faf4814d6d0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf3fe00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ba0 tx=0x7faf30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3002cd00 con 0x7faf4814d6d0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30004030 con 0x7faf4814d6d0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf48153950 con 0x7faf4814d6d0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30004420 con 0x7faf4814d6d0 2024-03-28T17:58:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.780+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf48153f10 con 0x7faf4814d6d0 2024-03-28T17:58:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.781+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf48073490 con 0x7faf4814d6d0 2024-03-28T17:58:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.782+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf3003d040 con 0x7faf4814d6d0 2024-03-28T17:58:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.783+0000 7faf44c00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 0x7faf2406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.783+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf300ac980 con 0x7faf4814d6d0 2024-03-28T17:58:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.783+0000 7faf47e00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 0x7faf2406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.784+0000 7faf47e00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 0x7faf2406e0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf38001820 tx=0x7faf38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.788+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faf30043020 con 0x7faf4814d6d0 2024-03-28T17:58:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.992+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4804f4e0 con 0x7faf4814d6d0 2024-03-28T17:58:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.994+0000 7faf44c00640 1 -- 172.21.15.62:0/2360846954 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7faf3007b3d0 con 0x7faf4814d6d0 2024-03-28T17:58:57.994 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:57.994 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.996+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 msgr2=0x7faf2406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.996+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 0x7faf2406e0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf38001820 tx=0x7faf38009040 comp rx=0 tx=0).stop 2024-03-28T17:58:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.996+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 msgr2=0x7faf48160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.996+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ba0 tx=0x7faf30004730 comp rx=0 tx=0).stop 2024-03-28T17:58:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 shutdown_connections 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faf2406bc00 0x7faf2406e0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf480cdee0 0x7faf480ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf4814d6d0 0x7faf48160880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 --2- 172.21.15.62:0/2360846954 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faf2c003f20 0x7faf48152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 >> 172.21.15.62:0/2360846954 conn(0x7faf480b1730 msgr2=0x7faf480717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 shutdown_connections 2024-03-28T17:58:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:57.997+0000 7faf4da00640 1 -- 172.21.15.62:0/2360846954 wait complete. 2024-03-28T17:58:58.017 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T17:58:58.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T17:58:58.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.230+0000 7f225ea00640 1 Processor -- start 2024-03-28T17:58:58.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.231+0000 7f225ea00640 1 -- start start 2024-03-28T17:58:58.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.232+0000 7f225ea00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.232+0000 7f225ea00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 0x7f2258071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f225ea00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22580b3700 con 0x7f22580b31f0 2024-03-28T17:58:58.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f225ea00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22580b39e0 con 0x7f225815ef60 2024-03-28T17:58:58.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f225ea00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22580b3cc0 con 0x7f2258144740 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f2257e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f225ca00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 0x7f2258071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f2257e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54026/0 (socket says 172.21.15.62:54026) 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.233+0000 7f2257e00640 1 -- 172.21.15.62:0/3831322205 learned_addr learned my addr 172.21.15.62:0/3831322205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.234+0000 7f2257e00640 1 -- 172.21.15.62:0/3831322205 >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 msgr2=0x7f2258071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.234+0000 7f2257e00640 1 --2- 172.21.15.62:0/3831322205 >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 0x7f2258071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.234+0000 7f2257e00640 1 -- 172.21.15.62:0/3831322205 >> v1:172.21.15.81:6789/0 conn(0x7f22580b31f0 legacy=0x7f225815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.234+0000 7f2257e00640 1 -- 172.21.15.62:0/3831322205 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22580b3fa0 con 0x7f2258144740 2024-03-28T17:58:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.234+0000 7f225ca00640 1 --2- 172.21.15.62:0/3831322205 >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 0x7f2258071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:58:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.235+0000 7f2257e00640 1 --2- 172.21.15.62:0/3831322205 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df4ad995d901b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.235+0000 7f2256a00640 1 -- 172.21.15.62:0/3831322205 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248034070 con 0x7f2258144740 2024-03-28T17:58:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.235+0000 7f2256a00640 1 -- 172.21.15.62:0/3831322205 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248030840 con 0x7f2258144740 2024-03-28T17:58:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.235+0000 7f2256a00640 1 -- 172.21.15.62:0/3831322205 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248030c10 con 0x7f2258144740 2024-03-28T17:58:58.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 -- 172.21.15.62:0/3831322205 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 msgr2=0x7f22580b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 --2- 172.21.15.62:0/3831322205 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802b980 comp rx=0 tx=0).stop 2024-03-28T17:58:58.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 -- 172.21.15.62:0/3831322205 shutdown_connections 2024-03-28T17:58:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 --2- 172.21.15.62:0/3831322205 >> v2:172.21.15.81:3300/0 conn(0x7f225815ef60 0x7f2258071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 --2- 172.21.15.62:0/3831322205 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f22580b2cb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 -- 172.21.15.62:0/3831322205 >> 172.21.15.62:0/3831322205 conn(0x7f2258066fb0 msgr2=0x7f22580673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 -- 172.21.15.62:0/3831322205 shutdown_connections 2024-03-28T17:58:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.236+0000 7f225ea00640 1 -- 172.21.15.62:0/3831322205 wait complete. 2024-03-28T17:58:58.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.239+0000 7f225ea00640 1 Processor -- start 2024-03-28T17:58:58.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.239+0000 7f225ea00640 1 -- start start 2024-03-28T17:58:58.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f225809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f225815ef60 0x7f225809d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22580716e0 con 0x7f22580b31f0 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2258071560 con 0x7f225815ef60 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f225ea00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2258071860 con 0x7f2258144740 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.240+0000 7f2257e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46772/0 (socket says 172.21.15.62:46772) 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 -- 172.21.15.62:0/4026948842 learned_addr learned my addr 172.21.15.62:0/4026948842 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f225ca00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f225815ef60 0x7f225809d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257400640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f225809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 -- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 msgr2=0x7f225809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f225809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 -- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f225815ef60 msgr2=0x7f225809d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f225815ef60 0x7f225809d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 -- 172.21.15.62:0/4026948842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225809d890 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257400640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f225809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.241+0000 7f2257e00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2248000c00 tx=0x7f2248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.242+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248048070 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.242+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248030640 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.242+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248034040 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.242+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225809db70 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.242+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22580ca490 con 0x7f22580b31f0 2024-03-28T17:58:58.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.243+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22580680c0 con 0x7f22580b31f0 2024-03-28T17:58:58.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.244+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2248037070 con 0x7f22580b31f0 2024-03-28T17:58:58.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.245+0000 7f224fe00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 0x7f223806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.245+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22480ac6f0 con 0x7f22580b31f0 2024-03-28T17:58:58.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.245+0000 7f2257400640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 0x7f223806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.247+0000 7f2257400640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 0x7f223806e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2244007fd0 tx=0x7f2244009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.250+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f224807b300 con 0x7f22580b31f0 2024-03-28T17:58:58.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.439+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f225806e2a0 con 0x7f223806bbd0 2024-03-28T17:58:58.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.461+0000 7f224fe00640 1 -- 172.21.15.62:0/4026948842 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f225806e2a0 con 0x7f223806bbd0 2024-03-28T17:58:58.465 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:58.466 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 msgr2=0x7f223806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 0x7f223806e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2244007fd0 tx=0x7f2244009040 comp rx=0 tx=0).stop 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 msgr2=0x7f22581f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2248000c00 tx=0x7f2248004730 comp rx=0 tx=0).stop 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 shutdown_connections 2024-03-28T17:58:58.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f223806bbd0 0x7f223806e090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f225815ef60 0x7f225809d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2258144740 0x7f225809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 --2- 172.21.15.62:0/4026948842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22580b31f0 0x7f22581f74a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 >> 172.21.15.62:0/4026948842 conn(0x7f2258066fb0 msgr2=0x7f2258067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 shutdown_connections 2024-03-28T17:58:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.466+0000 7f225ea00640 1 -- 172.21.15.62:0/4026948842 wait complete. 2024-03-28T17:58:58.484 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-28T17:58:58.305255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001666"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096023+0000","last_change":"2024-03-28T17:58:54.011100+0000","last_active":"2024-03-28T17:58:55.096023+0000","last_peered":"2024-03-28T17:58:55.096023+0000","last_clean":"2024-03-28T17:58:55.096023+0000","last_became_active":"2024-03-28T17:58:54.010769+0000","last_became_peered":"2024-03-28T17:58:54.010769+0000","last_unstale":"2024-03-28T17:58:55.096023+0000","last_undegraded":"2024-03-28T17:58:55.096023+0000","last_fullsized":"2024-03-28T17:58:55.096023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999288+0000","last_change":"2024-03-28T17:58:54.016056+0000","last_active":"2024-03-28T17:58:54.999288+0000","last_peered":"2024-03-28T17:58:54.999288+0000","last_clean":"2024-03-28T17:58:54.999288+0000","last_became_active":"2024-03-28T17:58:54.012351+0000","last_became_peered":"2024-03-28T17:58:54.012351+0000","last_unstale":"2024-03-28T17:58:54.999288+0000","last_undegraded":"2024-03-28T17:58:54.999288+0000","last_fullsized":"2024-03-28T17:58:54.999288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097234+0000","last_change":"2024-03-28T17:58:53.996524+0000","last_active":"2024-03-28T17:58:55.097234+0000","last_peered":"2024-03-28T17:58:55.097234+0000","last_clean":"2024-03-28T17:58:55.097234+0000","last_became_active":"2024-03-28T17:58:53.996197+0000","last_became_peered":"2024-03-28T17:58:53.996197+0000","last_unstale":"2024-03-28T17:58:55.097234+0000","last_undegraded":"2024-03-28T17:58:55.097234+0000","last_fullsized":"2024-03-28T17:58:55.097234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:16.038136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.001103+0000","last_change":"2024-03-28T17:58:54.009958+0000","last_active":"2024-03-28T17:58:55.001103+0000","last_peered":"2024-03-28T17:58:55.001103+0000","last_clean":"2024-03-28T17:58:55.001103+0000","last_became_active":"2024-03-28T17:58:54.009279+0000","last_became_peered":"2024-03-28T17:58:54.009279+0000","last_unstale":"2024-03-28T17:58:55.001103+0000","last_undegraded":"2024-03-28T17:58:55.001103+0000","last_fullsized":"2024-03-28T17:58:55.001103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095127+0000","last_change":"2024-03-28T17:58:54.007553+0000","last_active":"2024-03-28T17:58:55.095127+0000","last_peered":"2024-03-28T17:58:55.095127+0000","last_clean":"2024-03-28T17:58:55.095127+0000","last_became_active":"2024-03-28T17:58:54.007177+0000","last_became_peered":"2024-03-28T17:58:54.007177+0000","last_unstale":"2024-03-28T17:58:55.095127+0000","last_undegraded":"2024-03-28T17:58:55.095127+0000","last_fullsized":"2024-03-28T17:58:55.095127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999258+0000","last_change":"2024-03-28T17:58:53.998917+0000","last_active":"2024-03-28T17:58:54.999258+0000","last_peered":"2024-03-28T17:58:54.999258+0000","last_clean":"2024-03-28T17:58:54.999258+0000","last_became_active":"2024-03-28T17:58:53.998692+0000","last_became_peered":"2024-03-28T17:58:53.998692+0000","last_unstale":"2024-03-28T17:58:54.999258+0000","last_undegraded":"2024-03-28T17:58:54.999258+0000","last_fullsized":"2024-03-28T17:58:54.999258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094845+0000","last_change":"2024-03-28T17:58:54.007407+0000","last_active":"2024-03-28T17:58:55.094845+0000","last_peered":"2024-03-28T17:58:55.094845+0000","last_clean":"2024-03-28T17:58:55.094845+0000","last_became_active":"2024-03-28T17:58:54.007126+0000","last_became_peered":"2024-03-28T17:58:54.007126+0000","last_unstale":"2024-03-28T17:58:55.094845+0000","last_undegraded":"2024-03-28T17:58:55.094845+0000","last_fullsized":"2024-03-28T17:58:55.094845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094632+0000","last_change":"2024-03-28T17:58:54.021995+0000","last_active":"2024-03-28T17:58:55.094632+0000","last_peered":"2024-03-28T17:58:55.094632+0000","last_clean":"2024-03-28T17:58:55.094632+0000","last_became_active":"2024-03-28T17:58:54.021661+0000","last_became_peered":"2024-03-28T17:58:54.021661+0000","last_unstale":"2024-03-28T17:58:55.094632+0000","last_undegraded":"2024-03-28T17:58:55.094632+0000","last_fullsized":"2024-03-28T17:58:55.094632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096407+0000","last_change":"2024-03-28T17:58:54.014331+0000","last_active":"2024-03-28T17:58:55.096407+0000","last_peered":"2024-03-28T17:58:55.096407+0000","last_clean":"2024-03-28T17:58:55.096407+0000","last_became_active":"2024-03-28T17:58:54.014014+0000","last_became_peered":"2024-03-28T17:58:54.014014+0000","last_unstale":"2024-03-28T17:58:55.096407+0000","last_undegraded":"2024-03-28T17:58:55.096407+0000","last_fullsized":"2024-03-28T17:58:55.096407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019925+0000","last_change":"2024-03-28T17:58:54.013693+0000","last_active":"2024-03-28T17:58:55.019925+0000","last_peered":"2024-03-28T17:58:55.019925+0000","last_clean":"2024-03-28T17:58:55.019925+0000","last_became_active":"2024-03-28T17:58:54.013397+0000","last_became_peered":"2024-03-28T17:58:54.013397+0000","last_unstale":"2024-03-28T17:58:55.019925+0000","last_undegraded":"2024-03-28T17:58:55.019925+0000","last_fullsized":"2024-03-28T17:58:55.019925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019799+0000","last_change":"2024-03-28T17:58:54.014516+0000","last_active":"2024-03-28T17:58:55.019799+0000","last_peered":"2024-03-28T17:58:55.019799+0000","last_clean":"2024-03-28T17:58:55.019799+0000","last_became_active":"2024-03-28T17:58:54.014122+0000","last_became_peered":"2024-03-28T17:58:54.014122+0000","last_unstale":"2024-03-28T17:58:55.019799+0000","last_undegraded":"2024-03-28T17:58:55.019799+0000","last_fullsized":"2024-03-28T17:58:55.019799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097339+0000","last_change":"2024-03-28T17:58:54.015180+0000","last_active":"2024-03-28T17:58:55.097339+0000","last_peered":"2024-03-28T17:58:55.097339+0000","last_clean":"2024-03-28T17:58:55.097339+0000","last_became_active":"2024-03-28T17:58:54.008685+0000","last_became_peered":"2024-03-28T17:58:54.008685+0000","last_unstale":"2024-03-28T17:58:55.097339+0000","last_undegraded":"2024-03-28T17:58:55.097339+0000","last_fullsized":"2024-03-28T17:58:55.097339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095295+0000","last_change":"2024-03-28T17:58:54.012021+0000","last_active":"2024-03-28T17:58:55.095295+0000","last_peered":"2024-03-28T17:58:55.095295+0000","last_clean":"2024-03-28T17:58:55.095295+0000","last_became_active":"2024-03-28T17:58:54.011650+0000","last_became_peered":"2024-03-28T17:58:54.011650+0000","last_unstale":"2024-03-28T17:58:55.095295+0000","last_undegraded":"2024-03-28T17:58:55.095295+0000","last_fullsized":"2024-03-28T17:58:55.095295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097361+0000","last_change":"2024-03-28T17:58:54.011097+0000","last_active":"2024-03-28T17:58:55.097361+0000","last_peered":"2024-03-28T17:58:55.097361+0000","last_clean":"2024-03-28T17:58:55.097361+0000","last_became_active":"2024-03-28T17:58:54.010790+0000","last_became_peered":"2024-03-28T17:58:54.010790+0000","last_unstale":"2024-03-28T17:58:55.097361+0000","last_undegraded":"2024-03-28T17:58:55.097361+0000","last_fullsized":"2024-03-28T17:58:55.097361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:51:58.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996815+0000","last_change":"2024-03-28T17:58:54.022714+0000","last_active":"2024-03-28T17:58:54.996815+0000","last_peered":"2024-03-28T17:58:54.996815+0000","last_clean":"2024-03-28T17:58:54.996815+0000","last_became_active":"2024-03-28T17:58:54.022409+0000","last_became_peered":"2024-03-28T17:58:54.022409+0000","last_unstale":"2024-03-28T17:58:54.996815+0000","last_undegraded":"2024-03-28T17:58:54.996815+0000","last_fullsized":"2024-03-28T17:58:54.996815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999073+0000","last_change":"2024-03-28T17:58:54.005916+0000","last_active":"2024-03-28T17:58:54.999073+0000","last_peered":"2024-03-28T17:58:54.999073+0000","last_clean":"2024-03-28T17:58:54.999073+0000","last_became_active":"2024-03-28T17:58:54.005631+0000","last_became_peered":"2024-03-28T17:58:54.005631+0000","last_unstale":"2024-03-28T17:58:54.999073+0000","last_undegraded":"2024-03-28T17:58:54.999073+0000","last_fullsized":"2024-03-28T17:58:54.999073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000894+0000","last_change":"2024-03-28T17:58:54.005717+0000","last_active":"2024-03-28T17:58:55.000894+0000","last_peered":"2024-03-28T17:58:55.000894+0000","last_clean":"2024-03-28T17:58:55.000894+0000","last_became_active":"2024-03-28T17:58:54.005397+0000","last_became_peered":"2024-03-28T17:58:54.005397+0000","last_unstale":"2024-03-28T17:58:55.000894+0000","last_undegraded":"2024-03-28T17:58:55.000894+0000","last_fullsized":"2024-03-28T17:58:55.000894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997043+0000","last_change":"2024-03-28T17:58:54.009673+0000","last_active":"2024-03-28T17:58:54.997043+0000","last_peered":"2024-03-28T17:58:54.997043+0000","last_clean":"2024-03-28T17:58:54.997043+0000","last_became_active":"2024-03-28T17:58:54.009387+0000","last_became_peered":"2024-03-28T17:58:54.009387+0000","last_unstale":"2024-03-28T17:58:54.997043+0000","last_undegraded":"2024-03-28T17:58:54.997043+0000","last_fullsized":"2024-03-28T17:58:54.997043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:52.016086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095794+0000","last_change":"2024-03-28T17:58:54.003074+0000","last_active":"2024-03-28T17:58:55.095794+0000","last_peered":"2024-03-28T17:58:55.095794+0000","last_clean":"2024-03-28T17:58:55.095794+0000","last_became_active":"2024-03-28T17:58:54.002854+0000","last_became_peered":"2024-03-28T17:58:54.002854+0000","last_unstale":"2024-03-28T17:58:55.095794+0000","last_undegraded":"2024-03-28T17:58:55.095794+0000","last_fullsized":"2024-03-28T17:58:55.095794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019743+0000","last_change":"2024-03-28T17:58:54.022729+0000","last_active":"2024-03-28T17:58:55.019743+0000","last_peered":"2024-03-28T17:58:55.019743+0000","last_clean":"2024-03-28T17:58:55.019743+0000","last_became_active":"2024-03-28T17:58:54.022377+0000","last_became_peered":"2024-03-28T17:58:54.022377+0000","last_unstale":"2024-03-28T17:58:55.019743+0000","last_undegraded":"2024-03-28T17:58:55.019743+0000","last_fullsized":"2024-03-28T17:58:55.019743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098073+0000","last_change":"2024-03-28T17:58:54.005642+0000","last_active":"2024-03-28T17:58:55.098073+0000","last_peered":"2024-03-28T17:58:55.098073+0000","last_clean":"2024-03-28T17:58:55.098073+0000","last_became_active":"2024-03-28T17:58:54.005344+0000","last_became_peered":"2024-03-28T17:58:54.005344+0000","last_unstale":"2024-03-28T17:58:55.098073+0000","last_undegraded":"2024-03-28T17:58:55.098073+0000","last_fullsized":"2024-03-28T17:58:55.098073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999309+0000","last_change":"2024-03-28T17:58:54.005142+0000","last_active":"2024-03-28T17:58:54.999309+0000","last_peered":"2024-03-28T17:58:54.999309+0000","last_clean":"2024-03-28T17:58:54.999309+0000","last_became_active":"2024-03-28T17:58:54.003593+0000","last_became_peered":"2024-03-28T17:58:54.003593+0000","last_unstale":"2024-03-28T17:58:54.999309+0000","last_undegraded":"2024-03-28T17:58:54.999309+0000","last_fullsized":"2024-03-28T17:58:54.999309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095680+0000","last_change":"2024-03-28T17:58:54.020402+0000","last_active":"2024-03-28T17:58:55.095680+0000","last_peered":"2024-03-28T17:58:55.095680+0000","last_clean":"2024-03-28T17:58:55.095680+0000","last_became_active":"2024-03-28T17:58:54.020055+0000","last_became_peered":"2024-03-28T17:58:54.020055+0000","last_unstale":"2024-03-28T17:58:55.095680+0000","last_undegraded":"2024-03-28T17:58:55.095680+0000","last_fullsized":"2024-03-28T17:58:55.095680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999282+0000","last_change":"2024-03-28T17:58:54.012462+0000","last_active":"2024-03-28T17:58:54.999282+0000","last_peered":"2024-03-28T17:58:54.999282+0000","last_clean":"2024-03-28T17:58:54.999282+0000","last_became_active":"2024-03-28T17:58:54.011568+0000","last_became_peered":"2024-03-28T17:58:54.011568+0000","last_unstale":"2024-03-28T17:58:54.999282+0000","last_undegraded":"2024-03-28T17:58:54.999282+0000","last_fullsized":"2024-03-28T17:58:54.999282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095340+0000","last_change":"2024-03-28T17:58:54.023228+0000","last_active":"2024-03-28T17:58:55.095340+0000","last_peered":"2024-03-28T17:58:55.095340+0000","last_clean":"2024-03-28T17:58:55.095340+0000","last_became_active":"2024-03-28T17:58:54.022910+0000","last_became_peered":"2024-03-28T17:58:54.022910+0000","last_unstale":"2024-03-28T17:58:55.095340+0000","last_undegraded":"2024-03-28T17:58:55.095340+0000","last_fullsized":"2024-03-28T17:58:55.095340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019938+0000","last_change":"2024-03-28T17:58:54.006053+0000","last_active":"2024-03-28T17:58:55.019938+0000","last_peered":"2024-03-28T17:58:55.019938+0000","last_clean":"2024-03-28T17:58:55.019938+0000","last_became_active":"2024-03-28T17:58:54.005693+0000","last_became_peered":"2024-03-28T17:58:54.005693+0000","last_unstale":"2024-03-28T17:58:55.019938+0000","last_undegraded":"2024-03-28T17:58:55.019938+0000","last_fullsized":"2024-03-28T17:58:55.019938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998856+0000","last_change":"2024-03-28T17:58:54.013535+0000","last_active":"2024-03-28T17:58:54.998856+0000","last_peered":"2024-03-28T17:58:54.998856+0000","last_clean":"2024-03-28T17:58:54.998856+0000","last_became_active":"2024-03-28T17:58:54.011843+0000","last_became_peered":"2024-03-28T17:58:54.011843+0000","last_unstale":"2024-03-28T17:58:54.998856+0000","last_undegraded":"2024-03-28T17:58:54.998856+0000","last_fullsized":"2024-03-28T17:58:54.998856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095519+0000","last_change":"2024-03-28T17:58:54.022600+0000","last_active":"2024-03-28T17:58:55.095519+0000","last_peered":"2024-03-28T17:58:55.095519+0000","last_clean":"2024-03-28T17:58:55.095519+0000","last_became_active":"2024-03-28T17:58:54.022354+0000","last_became_peered":"2024-03-28T17:58:54.022354+0000","last_unstale":"2024-03-28T17:58:55.095519+0000","last_undegraded":"2024-03-28T17:58:55.095519+0000","last_fullsized":"2024-03-28T17:58:55.095519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097778+0000","last_change":"2024-03-28T17:58:54.005723+0000","last_active":"2024-03-28T17:58:55.097778+0000","last_peered":"2024-03-28T17:58:55.097778+0000","last_clean":"2024-03-28T17:58:55.097778+0000","last_became_active":"2024-03-28T17:58:54.005383+0000","last_became_peered":"2024-03-28T17:58:54.005383+0000","last_unstale":"2024-03-28T17:58:55.097778+0000","last_undegraded":"2024-03-28T17:58:55.097778+0000","last_fullsized":"2024-03-28T17:58:55.097778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:54.012921+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:54.012471+0000","last_became_peered":"2024-03-28T17:58:54.012471+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996217+0000","last_change":"2024-03-28T17:58:53.996340+0000","last_active":"2024-03-28T17:58:54.996217+0000","last_peered":"2024-03-28T17:58:54.996217+0000","last_clean":"2024-03-28T17:58:54.996217+0000","last_became_active":"2024-03-28T17:58:53.995914+0000","last_became_peered":"2024-03-28T17:58:53.995914+0000","last_unstale":"2024-03-28T17:58:54.996217+0000","last_undegraded":"2024-03-28T17:58:54.996217+0000","last_fullsized":"2024-03-28T17:58:54.996217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019333+0000","last_change":"2024-03-28T17:58:54.012238+0000","last_active":"2024-03-28T17:58:55.019333+0000","last_peered":"2024-03-28T17:58:55.019333+0000","last_clean":"2024-03-28T17:58:55.019333+0000","last_became_active":"2024-03-28T17:58:54.011952+0000","last_became_peered":"2024-03-28T17:58:54.011952+0000","last_unstale":"2024-03-28T17:58:55.019333+0000","last_undegraded":"2024-03-28T17:58:55.019333+0000","last_fullsized":"2024-03-28T17:58:55.019333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096509+0000","last_change":"2024-03-28T17:58:54.011087+0000","last_active":"2024-03-28T17:58:55.096509+0000","last_peered":"2024-03-28T17:58:55.096509+0000","last_clean":"2024-03-28T17:58:55.096509+0000","last_became_active":"2024-03-28T17:58:54.010787+0000","last_became_peered":"2024-03-28T17:58:54.010787+0000","last_unstale":"2024-03-28T17:58:55.096509+0000","last_undegraded":"2024-03-28T17:58:55.096509+0000","last_fullsized":"2024-03-28T17:58:55.096509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:23.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095155+0000","last_change":"2024-03-28T17:58:53.002153+0000","last_active":"2024-03-28T17:58:55.095155+0000","last_peered":"2024-03-28T17:58:55.095155+0000","last_clean":"2024-03-28T17:58:55.095155+0000","last_became_active":"2024-03-28T17:58:53.001844+0000","last_became_peered":"2024-03-28T17:58:53.001844+0000","last_unstale":"2024-03-28T17:58:55.095155+0000","last_undegraded":"2024-03-28T17:58:55.095155+0000","last_fullsized":"2024-03-28T17:58:55.095155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:38:14.392145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097946+0000","last_change":"2024-03-28T17:58:54.027047+0000","last_active":"2024-03-28T17:58:55.097946+0000","last_peered":"2024-03-28T17:58:55.097946+0000","last_clean":"2024-03-28T17:58:55.097946+0000","last_became_active":"2024-03-28T17:58:54.022997+0000","last_became_peered":"2024-03-28T17:58:54.022997+0000","last_unstale":"2024-03-28T17:58:55.097946+0000","last_undegraded":"2024-03-28T17:58:55.097946+0000","last_fullsized":"2024-03-28T17:58:55.097946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019270+0000","last_change":"2024-03-28T17:58:53.000389+0000","last_active":"2024-03-28T17:58:55.019270+0000","last_peered":"2024-03-28T17:58:55.019270+0000","last_clean":"2024-03-28T17:58:55.019270+0000","last_became_active":"2024-03-28T17:58:53.000086+0000","last_became_peered":"2024-03-28T17:58:53.000086+0000","last_unstale":"2024-03-28T17:58:55.019270+0000","last_undegraded":"2024-03-28T17:58:55.019270+0000","last_fullsized":"2024-03-28T17:58:55.019270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094817+0000","last_change":"2024-03-28T17:58:53.004667+0000","last_active":"2024-03-28T17:58:55.094817+0000","last_peered":"2024-03-28T17:58:55.094817+0000","last_clean":"2024-03-28T17:58:55.094817+0000","last_became_active":"2024-03-28T17:58:53.004337+0000","last_became_peered":"2024-03-28T17:58:53.004337+0000","last_unstale":"2024-03-28T17:58:55.094817+0000","last_undegraded":"2024-03-28T17:58:55.094817+0000","last_fullsized":"2024-03-28T17:58:55.094817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094868+0000","last_change":"2024-03-28T17:58:54.008588+0000","last_active":"2024-03-28T17:58:55.094868+0000","last_peered":"2024-03-28T17:58:55.094868+0000","last_clean":"2024-03-28T17:58:55.094868+0000","last_became_active":"2024-03-28T17:58:54.005942+0000","last_became_peered":"2024-03-28T17:58:54.005942+0000","last_unstale":"2024-03-28T17:58:55.094868+0000","last_undegraded":"2024-03-28T17:58:55.094868+0000","last_fullsized":"2024-03-28T17:58:55.094868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999261+0000","last_change":"2024-03-28T17:58:53.003611+0000","last_active":"2024-03-28T17:58:54.999261+0000","last_peered":"2024-03-28T17:58:54.999261+0000","last_clean":"2024-03-28T17:58:54.999261+0000","last_became_active":"2024-03-28T17:58:53.003272+0000","last_became_peered":"2024-03-28T17:58:53.003272+0000","last_unstale":"2024-03-28T17:58:54.999261+0000","last_undegraded":"2024-03-28T17:58:54.999261+0000","last_fullsized":"2024-03-28T17:58:54.999261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996592+0000","last_change":"2024-03-28T17:58:54.013586+0000","last_active":"2024-03-28T17:58:54.996592+0000","last_peered":"2024-03-28T17:58:54.996592+0000","last_clean":"2024-03-28T17:58:54.996592+0000","last_became_active":"2024-03-28T17:58:54.013293+0000","last_became_peered":"2024-03-28T17:58:54.013293+0000","last_unstale":"2024-03-28T17:58:54.996592+0000","last_undegraded":"2024-03-28T17:58:54.996592+0000","last_fullsized":"2024-03-28T17:58:54.996592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019821+0000","last_change":"2024-03-28T17:58:53.005607+0000","last_active":"2024-03-28T17:58:55.019821+0000","last_peered":"2024-03-28T17:58:55.019821+0000","last_clean":"2024-03-28T17:58:55.019821+0000","last_became_active":"2024-03-28T17:58:53.005306+0000","last_became_peered":"2024-03-28T17:58:53.005306+0000","last_unstale":"2024-03-28T17:58:55.019821+0000","last_undegraded":"2024-03-28T17:58:55.019821+0000","last_fullsized":"2024-03-28T17:58:55.019821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000404+0000","last_change":"2024-03-28T17:58:54.006871+0000","last_active":"2024-03-28T17:58:55.000404+0000","last_peered":"2024-03-28T17:58:55.000404+0000","last_clean":"2024-03-28T17:58:55.000404+0000","last_became_active":"2024-03-28T17:58:54.006521+0000","last_became_peered":"2024-03-28T17:58:54.006521+0000","last_unstale":"2024-03-28T17:58:55.000404+0000","last_undegraded":"2024-03-28T17:58:55.000404+0000","last_fullsized":"2024-03-28T17:58:55.000404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000147+0000","last_change":"2024-03-28T17:58:53.008708+0000","last_active":"2024-03-28T17:58:55.000147+0000","last_peered":"2024-03-28T17:58:55.000147+0000","last_clean":"2024-03-28T17:58:55.000147+0000","last_became_active":"2024-03-28T17:58:53.008424+0000","last_became_peered":"2024-03-28T17:58:53.008424+0000","last_unstale":"2024-03-28T17:58:55.000147+0000","last_undegraded":"2024-03-28T17:58:55.000147+0000","last_fullsized":"2024-03-28T17:58:55.000147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098103+0000","last_change":"2024-03-28T17:58:54.007645+0000","last_active":"2024-03-28T17:58:55.098103+0000","last_peered":"2024-03-28T17:58:55.098103+0000","last_clean":"2024-03-28T17:58:55.098103+0000","last_became_active":"2024-03-28T17:58:54.007359+0000","last_became_peered":"2024-03-28T17:58:54.007359+0000","last_unstale":"2024-03-28T17:58:55.098103+0000","last_undegraded":"2024-03-28T17:58:55.098103+0000","last_fullsized":"2024-03-28T17:58:55.098103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998573+0000","last_change":"2024-03-28T17:58:53.000532+0000","last_active":"2024-03-28T17:58:54.998573+0000","last_peered":"2024-03-28T17:58:54.998573+0000","last_clean":"2024-03-28T17:58:54.998573+0000","last_became_active":"2024-03-28T17:58:53.000238+0000","last_became_peered":"2024-03-28T17:58:53.000238+0000","last_unstale":"2024-03-28T17:58:54.998573+0000","last_undegraded":"2024-03-28T17:58:54.998573+0000","last_fullsized":"2024-03-28T17:58:54.998573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097269+0000","last_change":"2024-03-28T17:58:54.011919+0000","last_active":"2024-03-28T17:58:55.097269+0000","last_peered":"2024-03-28T17:58:55.097269+0000","last_clean":"2024-03-28T17:58:55.097269+0000","last_became_active":"2024-03-28T17:58:54.008256+0000","last_became_peered":"2024-03-28T17:58:54.008256+0000","last_unstale":"2024-03-28T17:58:55.097269+0000","last_undegraded":"2024-03-28T17:58:55.097269+0000","last_fullsized":"2024-03-28T17:58:55.097269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:16.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019656+0000","last_change":"2024-03-28T17:58:53.004395+0000","last_active":"2024-03-28T17:58:55.019656+0000","last_peered":"2024-03-28T17:58:55.019656+0000","last_clean":"2024-03-28T17:58:55.019656+0000","last_became_active":"2024-03-28T17:58:52.999296+0000","last_became_peered":"2024-03-28T17:58:52.999296+0000","last_unstale":"2024-03-28T17:58:55.019656+0000","last_undegraded":"2024-03-28T17:58:55.019656+0000","last_fullsized":"2024-03-28T17:58:55.019656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095473+0000","last_change":"2024-03-28T17:58:54.004875+0000","last_active":"2024-03-28T17:58:55.095473+0000","last_peered":"2024-03-28T17:58:55.095473+0000","last_clean":"2024-03-28T17:58:55.095473+0000","last_became_active":"2024-03-28T17:58:54.004597+0000","last_became_peered":"2024-03-28T17:58:54.004597+0000","last_unstale":"2024-03-28T17:58:55.095473+0000","last_undegraded":"2024-03-28T17:58:55.095473+0000","last_fullsized":"2024-03-28T17:58:55.095473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098023+0000","last_change":"2024-03-28T17:58:53.009461+0000","last_active":"2024-03-28T17:58:55.098023+0000","last_peered":"2024-03-28T17:58:55.098023+0000","last_clean":"2024-03-28T17:58:55.098023+0000","last_became_active":"2024-03-28T17:58:53.008867+0000","last_became_peered":"2024-03-28T17:58:53.008867+0000","last_unstale":"2024-03-28T17:58:55.098023+0000","last_undegraded":"2024-03-28T17:58:55.098023+0000","last_fullsized":"2024-03-28T17:58:55.098023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:57:02.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098575+0000","last_change":"2024-03-28T17:58:54.001368+0000","last_active":"2024-03-28T17:58:55.098575+0000","last_peered":"2024-03-28T17:58:55.098575+0000","last_clean":"2024-03-28T17:58:55.098575+0000","last_became_active":"2024-03-28T17:58:54.000744+0000","last_became_peered":"2024-03-28T17:58:54.000744+0000","last_unstale":"2024-03-28T17:58:55.098575+0000","last_undegraded":"2024-03-28T17:58:55.098575+0000","last_fullsized":"2024-03-28T17:58:55.098575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999461+0000","last_change":"2024-03-28T17:58:53.007374+0000","last_active":"2024-03-28T17:58:54.999461+0000","last_peered":"2024-03-28T17:58:54.999461+0000","last_clean":"2024-03-28T17:58:54.999461+0000","last_became_active":"2024-03-28T17:58:53.005001+0000","last_became_peered":"2024-03-28T17:58:53.005001+0000","last_unstale":"2024-03-28T17:58:54.999461+0000","last_undegraded":"2024-03-28T17:58:54.999461+0000","last_fullsized":"2024-03-28T17:58:54.999461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095283+0000","last_change":"2024-03-28T17:58:54.020789+0000","last_active":"2024-03-28T17:58:55.095283+0000","last_peered":"2024-03-28T17:58:55.095283+0000","last_clean":"2024-03-28T17:58:55.095283+0000","last_became_active":"2024-03-28T17:58:54.020338+0000","last_became_peered":"2024-03-28T17:58:54.020338+0000","last_unstale":"2024-03-28T17:58:55.095283+0000","last_undegraded":"2024-03-28T17:58:55.095283+0000","last_fullsized":"2024-03-28T17:58:55.095283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019218+0000","last_change":"2024-03-28T17:58:53.001274+0000","last_active":"2024-03-28T17:58:55.019218+0000","last_peered":"2024-03-28T17:58:55.019218+0000","last_clean":"2024-03-28T17:58:55.019218+0000","last_became_active":"2024-03-28T17:58:53.000949+0000","last_became_peered":"2024-03-28T17:58:53.000949+0000","last_unstale":"2024-03-28T17:58:55.019218+0000","last_undegraded":"2024-03-28T17:58:55.019218+0000","last_fullsized":"2024-03-28T17:58:55.019218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019704+0000","last_change":"2024-03-28T17:58:54.009630+0000","last_active":"2024-03-28T17:58:55.019704+0000","last_peered":"2024-03-28T17:58:55.019704+0000","last_clean":"2024-03-28T17:58:55.019704+0000","last_became_active":"2024-03-28T17:58:54.009072+0000","last_became_peered":"2024-03-28T17:58:54.009072+0000","last_unstale":"2024-03-28T17:58:55.019704+0000","last_undegraded":"2024-03-28T17:58:55.019704+0000","last_fullsized":"2024-03-28T17:58:55.019704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.995993+0000","last_change":"2024-03-28T17:58:53.002389+0000","last_active":"2024-03-28T17:58:54.995993+0000","last_peered":"2024-03-28T17:58:54.995993+0000","last_clean":"2024-03-28T17:58:54.995993+0000","last_became_active":"2024-03-28T17:58:53.001859+0000","last_became_peered":"2024-03-28T17:58:53.001859+0000","last_unstale":"2024-03-28T17:58:54.995993+0000","last_undegraded":"2024-03-28T17:58:54.995993+0000","last_fullsized":"2024-03-28T17:58:54.995993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095130+0000","last_change":"2024-03-28T17:58:54.022077+0000","last_active":"2024-03-28T17:58:55.095130+0000","last_peered":"2024-03-28T17:58:55.095130+0000","last_clean":"2024-03-28T17:58:55.095130+0000","last_became_active":"2024-03-28T17:58:54.021287+0000","last_became_peered":"2024-03-28T17:58:54.021287+0000","last_unstale":"2024-03-28T17:58:55.095130+0000","last_undegraded":"2024-03-28T17:58:55.095130+0000","last_fullsized":"2024-03-28T17:58:55.095130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097510+0000","last_change":"2024-03-28T17:58:53.002001+0000","last_active":"2024-03-28T17:58:55.097510+0000","last_peered":"2024-03-28T17:58:55.097510+0000","last_clean":"2024-03-28T17:58:55.097510+0000","last_became_active":"2024-03-28T17:58:53.001734+0000","last_became_peered":"2024-03-28T17:58:53.001734+0000","last_unstale":"2024-03-28T17:58:55.097510+0000","last_undegraded":"2024-03-28T17:58:55.097510+0000","last_fullsized":"2024-03-28T17:58:55.097510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:13:29.578775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095671+0000","last_change":"2024-03-28T17:58:54.021625+0000","last_active":"2024-03-28T17:58:55.095671+0000","last_peered":"2024-03-28T17:58:55.095671+0000","last_clean":"2024-03-28T17:58:55.095671+0000","last_became_active":"2024-03-28T17:58:54.020370+0000","last_became_peered":"2024-03-28T17:58:54.020370+0000","last_unstale":"2024-03-28T17:58:55.095671+0000","last_undegraded":"2024-03-28T17:58:55.095671+0000","last_fullsized":"2024-03-28T17:58:55.095671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997096+0000","last_change":"2024-03-28T17:58:52.997706+0000","last_active":"2024-03-28T17:58:54.997096+0000","last_peered":"2024-03-28T17:58:54.997096+0000","last_clean":"2024-03-28T17:58:54.997096+0000","last_became_active":"2024-03-28T17:58:52.997002+0000","last_became_peered":"2024-03-28T17:58:52.997002+0000","last_unstale":"2024-03-28T17:58:54.997096+0000","last_undegraded":"2024-03-28T17:58:54.997096+0000","last_fullsized":"2024-03-28T17:58:54.997096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096799+0000","last_change":"2024-03-28T17:58:54.021721+0000","last_active":"2024-03-28T17:58:55.096799+0000","last_peered":"2024-03-28T17:58:55.096799+0000","last_clean":"2024-03-28T17:58:55.096799+0000","last_became_active":"2024-03-28T17:58:54.020973+0000","last_became_peered":"2024-03-28T17:58:54.020973+0000","last_unstale":"2024-03-28T17:58:55.096799+0000","last_undegraded":"2024-03-28T17:58:55.096799+0000","last_fullsized":"2024-03-28T17:58:55.096799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:29:06.711377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998981+0000","last_change":"2024-03-28T17:58:53.007402+0000","last_active":"2024-03-28T17:58:54.998981+0000","last_peered":"2024-03-28T17:58:54.998981+0000","last_clean":"2024-03-28T17:58:54.998981+0000","last_became_active":"2024-03-28T17:58:53.007131+0000","last_became_peered":"2024-03-28T17:58:53.007131+0000","last_unstale":"2024-03-28T17:58:54.998981+0000","last_undegraded":"2024-03-28T17:58:54.998981+0000","last_fullsized":"2024-03-28T17:58:54.998981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000866+0000","last_change":"2024-03-28T17:58:54.009567+0000","last_active":"2024-03-28T17:58:55.000866+0000","last_peered":"2024-03-28T17:58:55.000866+0000","last_clean":"2024-03-28T17:58:55.000866+0000","last_became_active":"2024-03-28T17:58:54.008926+0000","last_became_peered":"2024-03-28T17:58:54.008926+0000","last_unstale":"2024-03-28T17:58:55.000866+0000","last_undegraded":"2024-03-28T17:58:55.000866+0000","last_fullsized":"2024-03-28T17:58:55.000866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996703+0000","last_change":"2024-03-28T17:58:52.995453+0000","last_active":"2024-03-28T17:58:54.996703+0000","last_peered":"2024-03-28T17:58:54.996703+0000","last_clean":"2024-03-28T17:58:54.996703+0000","last_became_active":"2024-03-28T17:58:52.995177+0000","last_became_peered":"2024-03-28T17:58:52.995177+0000","last_unstale":"2024-03-28T17:58:54.996703+0000","last_undegraded":"2024-03-28T17:58:54.996703+0000","last_fullsized":"2024-03-28T17:58:54.996703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095614+0000","last_change":"2024-03-28T17:58:54.004742+0000","last_active":"2024-03-28T17:58:55.095614+0000","last_peered":"2024-03-28T17:58:55.095614+0000","last_clean":"2024-03-28T17:58:55.095614+0000","last_became_active":"2024-03-28T17:58:54.004434+0000","last_became_peered":"2024-03-28T17:58:54.004434+0000","last_unstale":"2024-03-28T17:58:55.095614+0000","last_undegraded":"2024-03-28T17:58:55.095614+0000","last_fullsized":"2024-03-28T17:58:55.095614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094833+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T17:58:55.094833+0000","last_peered":"2024-03-28T17:58:55.094833+0000","last_clean":"2024-03-28T17:58:55.094833+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T17:58:55.094833+0000","last_undegraded":"2024-03-28T17:58:55.094833+0000","last_fullsized":"2024-03-28T17:58:55.094833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999042+0000","last_change":"2024-03-28T17:58:52.995644+0000","last_active":"2024-03-28T17:58:54.999042+0000","last_peered":"2024-03-28T17:58:54.999042+0000","last_clean":"2024-03-28T17:58:54.999042+0000","last_became_active":"2024-03-28T17:58:52.995196+0000","last_became_peered":"2024-03-28T17:58:52.995196+0000","last_unstale":"2024-03-28T17:58:54.999042+0000","last_undegraded":"2024-03-28T17:58:54.999042+0000","last_fullsized":"2024-03-28T17:58:54.999042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095066+0000","last_change":"2024-03-28T17:58:54.015875+0000","last_active":"2024-03-28T17:58:55.095066+0000","last_peered":"2024-03-28T17:58:55.095066+0000","last_clean":"2024-03-28T17:58:55.095066+0000","last_became_active":"2024-03-28T17:58:54.014187+0000","last_became_peered":"2024-03-28T17:58:54.014187+0000","last_unstale":"2024-03-28T17:58:55.095066+0000","last_undegraded":"2024-03-28T17:58:55.095066+0000","last_fullsized":"2024-03-28T17:58:55.095066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094699+0000","last_change":"2024-03-28T17:58:53.010263+0000","last_active":"2024-03-28T17:58:55.094699+0000","last_peered":"2024-03-28T17:58:55.094699+0000","last_clean":"2024-03-28T17:58:55.094699+0000","last_became_active":"2024-03-28T17:58:53.009990+0000","last_became_peered":"2024-03-28T17:58:53.009990+0000","last_unstale":"2024-03-28T17:58:55.094699+0000","last_undegraded":"2024-03-28T17:58:55.094699+0000","last_fullsized":"2024-03-28T17:58:55.094699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097226+0000","last_change":"2024-03-28T17:58:54.022349+0000","last_active":"2024-03-28T17:58:55.097226+0000","last_peered":"2024-03-28T17:58:55.097226+0000","last_clean":"2024-03-28T17:58:55.097226+0000","last_became_active":"2024-03-28T17:58:54.021997+0000","last_became_peered":"2024-03-28T17:58:54.021997+0000","last_unstale":"2024-03-28T17:58:55.097226+0000","last_undegraded":"2024-03-28T17:58:55.097226+0000","last_fullsized":"2024-03-28T17:58:55.097226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:20.314101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996301+0000","last_change":"2024-03-28T17:58:52.994588+0000","last_active":"2024-03-28T17:58:54.996301+0000","last_peered":"2024-03-28T17:58:54.996301+0000","last_clean":"2024-03-28T17:58:54.996301+0000","last_became_active":"2024-03-28T17:58:52.994338+0000","last_became_peered":"2024-03-28T17:58:52.994338+0000","last_unstale":"2024-03-28T17:58:54.996301+0000","last_undegraded":"2024-03-28T17:58:54.996301+0000","last_fullsized":"2024-03-28T17:58:54.996301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997461+0000","last_change":"2024-03-28T17:58:53.998446+0000","last_active":"2024-03-28T17:58:54.997461+0000","last_peered":"2024-03-28T17:58:54.997461+0000","last_clean":"2024-03-28T17:58:54.997461+0000","last_became_active":"2024-03-28T17:58:53.998163+0000","last_became_peered":"2024-03-28T17:58:53.998163+0000","last_unstale":"2024-03-28T17:58:54.997461+0000","last_undegraded":"2024-03-28T17:58:54.997461+0000","last_fullsized":"2024-03-28T17:58:54.997461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:52.998174+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:52.997265+0000","last_became_peered":"2024-03-28T17:58:52.997265+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096877+0000","last_change":"2024-03-28T17:58:54.017676+0000","last_active":"2024-03-28T17:58:55.096877+0000","last_peered":"2024-03-28T17:58:55.096877+0000","last_clean":"2024-03-28T17:58:55.096877+0000","last_became_active":"2024-03-28T17:58:54.016900+0000","last_became_peered":"2024-03-28T17:58:54.016900+0000","last_unstale":"2024-03-28T17:58:55.096877+0000","last_undegraded":"2024-03-28T17:58:55.096877+0000","last_fullsized":"2024-03-28T17:58:55.096877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:45.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998525+0000","last_change":"2024-03-28T17:58:52.997517+0000","last_active":"2024-03-28T17:58:54.998525+0000","last_peered":"2024-03-28T17:58:54.998525+0000","last_clean":"2024-03-28T17:58:54.998525+0000","last_became_active":"2024-03-28T17:58:52.997207+0000","last_became_peered":"2024-03-28T17:58:52.997207+0000","last_unstale":"2024-03-28T17:58:54.998525+0000","last_undegraded":"2024-03-28T17:58:54.998525+0000","last_fullsized":"2024-03-28T17:58:54.998525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999951+0000","last_change":"2024-03-28T17:58:54.011090+0000","last_active":"2024-03-28T17:58:54.999951+0000","last_peered":"2024-03-28T17:58:54.999951+0000","last_clean":"2024-03-28T17:58:54.999951+0000","last_became_active":"2024-03-28T17:58:54.009864+0000","last_became_peered":"2024-03-28T17:58:54.009864+0000","last_unstale":"2024-03-28T17:58:54.999951+0000","last_undegraded":"2024-03-28T17:58:54.999951+0000","last_fullsized":"2024-03-28T17:58:54.999951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095825+0000","last_change":"2024-03-28T17:58:53.011734+0000","last_active":"2024-03-28T17:58:55.095825+0000","last_peered":"2024-03-28T17:58:55.095825+0000","last_clean":"2024-03-28T17:58:55.095825+0000","last_became_active":"2024-03-28T17:58:53.011209+0000","last_became_peered":"2024-03-28T17:58:53.011209+0000","last_unstale":"2024-03-28T17:58:55.095825+0000","last_undegraded":"2024-03-28T17:58:55.095825+0000","last_fullsized":"2024-03-28T17:58:55.095825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997836+0000","last_change":"2024-03-28T17:58:54.015424+0000","last_active":"2024-03-28T17:58:54.997836+0000","last_peered":"2024-03-28T17:58:54.997836+0000","last_clean":"2024-03-28T17:58:54.997836+0000","last_became_active":"2024-03-28T17:58:54.011122+0000","last_became_peered":"2024-03-28T17:58:54.011122+0000","last_unstale":"2024-03-28T17:58:54.997836+0000","last_undegraded":"2024-03-28T17:58:54.997836+0000","last_fullsized":"2024-03-28T17:58:54.997836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999477+0000","last_change":"2024-03-28T17:58:53.008757+0000","last_active":"2024-03-28T17:58:54.999477+0000","last_peered":"2024-03-28T17:58:54.999477+0000","last_clean":"2024-03-28T17:58:54.999477+0000","last_became_active":"2024-03-28T17:58:53.008455+0000","last_became_peered":"2024-03-28T17:58:53.008455+0000","last_unstale":"2024-03-28T17:58:54.999477+0000","last_undegraded":"2024-03-28T17:58:54.999477+0000","last_fullsized":"2024-03-28T17:58:54.999477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096368+0000","last_change":"2024-03-28T17:58:54.017029+0000","last_active":"2024-03-28T17:58:55.096368+0000","last_peered":"2024-03-28T17:58:55.096368+0000","last_clean":"2024-03-28T17:58:55.096368+0000","last_became_active":"2024-03-28T17:58:54.014826+0000","last_became_peered":"2024-03-28T17:58:54.014826+0000","last_unstale":"2024-03-28T17:58:55.096368+0000","last_undegraded":"2024-03-28T17:58:55.096368+0000","last_fullsized":"2024-03-28T17:58:55.096368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019932+0000","last_change":"2024-03-28T17:58:53.005570+0000","last_active":"2024-03-28T17:58:55.019932+0000","last_peered":"2024-03-28T17:58:55.019932+0000","last_clean":"2024-03-28T17:58:55.019932+0000","last_became_active":"2024-03-28T17:58:53.005285+0000","last_became_peered":"2024-03-28T17:58:53.005285+0000","last_unstale":"2024-03-28T17:58:55.019932+0000","last_undegraded":"2024-03-28T17:58:55.019932+0000","last_fullsized":"2024-03-28T17:58:55.019932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098116+0000","last_change":"2024-03-28T17:58:54.018755+0000","last_active":"2024-03-28T17:58:55.098116+0000","last_peered":"2024-03-28T17:58:55.098116+0000","last_clean":"2024-03-28T17:58:55.098116+0000","last_became_active":"2024-03-28T17:58:54.018403+0000","last_became_peered":"2024-03-28T17:58:54.018403+0000","last_unstale":"2024-03-28T17:58:55.098116+0000","last_undegraded":"2024-03-28T17:58:55.098116+0000","last_fullsized":"2024-03-28T17:58:55.098116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095055+0000","last_change":"2024-03-28T17:58:53.008509+0000","last_active":"2024-03-28T17:58:55.095055+0000","last_peered":"2024-03-28T17:58:55.095055+0000","last_clean":"2024-03-28T17:58:55.095055+0000","last_became_active":"2024-03-28T17:58:53.008065+0000","last_became_peered":"2024-03-28T17:58:53.008065+0000","last_unstale":"2024-03-28T17:58:55.095055+0000","last_undegraded":"2024-03-28T17:58:55.095055+0000","last_fullsized":"2024-03-28T17:58:55.095055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095224+0000","last_change":"2024-03-28T17:58:54.018790+0000","last_active":"2024-03-28T17:58:55.095224+0000","last_peered":"2024-03-28T17:58:55.095224+0000","last_clean":"2024-03-28T17:58:55.095224+0000","last_became_active":"2024-03-28T17:58:54.018306+0000","last_became_peered":"2024-03-28T17:58:54.018306+0000","last_unstale":"2024-03-28T17:58:55.095224+0000","last_undegraded":"2024-03-28T17:58:55.095224+0000","last_fullsized":"2024-03-28T17:58:55.095224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:02.245546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996892+0000","last_change":"2024-03-28T17:58:53.007176+0000","last_active":"2024-03-28T17:58:54.996892+0000","last_peered":"2024-03-28T17:58:54.996892+0000","last_clean":"2024-03-28T17:58:54.996892+0000","last_became_active":"2024-03-28T17:58:53.002876+0000","last_became_peered":"2024-03-28T17:58:53.002876+0000","last_unstale":"2024-03-28T17:58:54.996892+0000","last_undegraded":"2024-03-28T17:58:54.996892+0000","last_fullsized":"2024-03-28T17:58:54.996892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095113+0000","last_change":"2024-03-28T17:58:54.021263+0000","last_active":"2024-03-28T17:58:55.095113+0000","last_peered":"2024-03-28T17:58:55.095113+0000","last_clean":"2024-03-28T17:58:55.095113+0000","last_became_active":"2024-03-28T17:58:54.015718+0000","last_became_peered":"2024-03-28T17:58:54.015718+0000","last_unstale":"2024-03-28T17:58:55.095113+0000","last_undegraded":"2024-03-28T17:58:55.095113+0000","last_fullsized":"2024-03-28T17:58:55.095113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997526+0000","last_change":"2024-03-28T17:58:53.000138+0000","last_active":"2024-03-28T17:58:54.997526+0000","last_peered":"2024-03-28T17:58:54.997526+0000","last_clean":"2024-03-28T17:58:54.997526+0000","last_became_active":"2024-03-28T17:58:52.999805+0000","last_became_peered":"2024-03-28T17:58:52.999805+0000","last_unstale":"2024-03-28T17:58:54.997526+0000","last_undegraded":"2024-03-28T17:58:54.997526+0000","last_fullsized":"2024-03-28T17:58:54.997526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095084+0000","last_change":"2024-03-28T17:58:54.006123+0000","last_active":"2024-03-28T17:58:55.095084+0000","last_peered":"2024-03-28T17:58:55.095084+0000","last_clean":"2024-03-28T17:58:55.095084+0000","last_became_active":"2024-03-28T17:58:54.005714+0000","last_became_peered":"2024-03-28T17:58:54.005714+0000","last_unstale":"2024-03-28T17:58:55.095084+0000","last_undegraded":"2024-03-28T17:58:55.095084+0000","last_fullsized":"2024-03-28T17:58:55.095084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095450+0000","last_change":"2024-03-28T17:58:53.009871+0000","last_active":"2024-03-28T17:58:55.095450+0000","last_peered":"2024-03-28T17:58:55.095450+0000","last_clean":"2024-03-28T17:58:55.095450+0000","last_became_active":"2024-03-28T17:58:53.009550+0000","last_became_peered":"2024-03-28T17:58:53.009550+0000","last_unstale":"2024-03-28T17:58:55.095450+0000","last_undegraded":"2024-03-28T17:58:55.095450+0000","last_fullsized":"2024-03-28T17:58:55.095450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:03.353429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019393+0000","last_change":"2024-03-28T17:58:54.020986+0000","last_active":"2024-03-28T17:58:55.019393+0000","last_peered":"2024-03-28T17:58:55.019393+0000","last_clean":"2024-03-28T17:58:55.019393+0000","last_became_active":"2024-03-28T17:58:54.020717+0000","last_became_peered":"2024-03-28T17:58:54.020717+0000","last_unstale":"2024-03-28T17:58:55.019393+0000","last_undegraded":"2024-03-28T17:58:55.019393+0000","last_fullsized":"2024-03-28T17:58:55.019393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999845+0000","last_change":"2024-03-28T17:58:54.022957+0000","last_active":"2024-03-28T17:58:54.999845+0000","last_peered":"2024-03-28T17:58:54.999845+0000","last_clean":"2024-03-28T17:58:54.999845+0000","last_became_active":"2024-03-28T17:58:54.022234+0000","last_became_peered":"2024-03-28T17:58:54.022234+0000","last_unstale":"2024-03-28T17:58:54.999845+0000","last_undegraded":"2024-03-28T17:58:54.999845+0000","last_fullsized":"2024-03-28T17:58:54.999845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097078+0000","last_change":"2024-03-28T17:58:53.010194+0000","last_active":"2024-03-28T17:58:55.097078+0000","last_peered":"2024-03-28T17:58:55.097078+0000","last_clean":"2024-03-28T17:58:55.097078+0000","last_became_active":"2024-03-28T17:58:53.009750+0000","last_became_peered":"2024-03-28T17:58:53.009750+0000","last_unstale":"2024-03-28T17:58:55.097078+0000","last_undegraded":"2024-03-28T17:58:55.097078+0000","last_fullsized":"2024-03-28T17:58:55.097078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000545+0000","last_change":"2024-03-28T17:58:54.016961+0000","last_active":"2024-03-28T17:58:55.000545+0000","last_peered":"2024-03-28T17:58:55.000545+0000","last_clean":"2024-03-28T17:58:55.000545+0000","last_became_active":"2024-03-28T17:58:54.016695+0000","last_became_peered":"2024-03-28T17:58:54.016695+0000","last_unstale":"2024-03-28T17:58:55.000545+0000","last_undegraded":"2024-03-28T17:58:55.000545+0000","last_fullsized":"2024-03-28T17:58:55.000545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095647+0000","last_change":"2024-03-28T17:58:53.011900+0000","last_active":"2024-03-28T17:58:55.095647+0000","last_peered":"2024-03-28T17:58:55.095647+0000","last_clean":"2024-03-28T17:58:55.095647+0000","last_became_active":"2024-03-28T17:58:53.011463+0000","last_became_peered":"2024-03-28T17:58:53.011463+0000","last_unstale":"2024-03-28T17:58:55.095647+0000","last_undegraded":"2024-03-28T17:58:55.095647+0000","last_fullsized":"2024-03-28T17:58:55.095647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098095+0000","last_change":"2024-03-28T17:58:54.010789+0000","last_active":"2024-03-28T17:58:55.098095+0000","last_peered":"2024-03-28T17:58:55.098095+0000","last_clean":"2024-03-28T17:58:55.098095+0000","last_became_active":"2024-03-28T17:58:54.010520+0000","last_became_peered":"2024-03-28T17:58:54.010520+0000","last_unstale":"2024-03-28T17:58:55.098095+0000","last_undegraded":"2024-03-28T17:58:55.098095+0000","last_fullsized":"2024-03-28T17:58:55.098095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095653+0000","last_change":"2024-03-28T17:58:52.999999+0000","last_active":"2024-03-28T17:58:55.095653+0000","last_peered":"2024-03-28T17:58:55.095653+0000","last_clean":"2024-03-28T17:58:55.095653+0000","last_became_active":"2024-03-28T17:58:52.999659+0000","last_became_peered":"2024-03-28T17:58:52.999659+0000","last_unstale":"2024-03-28T17:58:55.095653+0000","last_undegraded":"2024-03-28T17:58:55.095653+0000","last_fullsized":"2024-03-28T17:58:55.095653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095619+0000","last_change":"2024-03-28T17:58:54.010224+0000","last_active":"2024-03-28T17:58:55.095619+0000","last_peered":"2024-03-28T17:58:55.095619+0000","last_clean":"2024-03-28T17:58:55.095619+0000","last_became_active":"2024-03-28T17:58:54.008396+0000","last_became_peered":"2024-03-28T17:58:54.008396+0000","last_unstale":"2024-03-28T17:58:55.095619+0000","last_undegraded":"2024-03-28T17:58:55.095619+0000","last_fullsized":"2024-03-28T17:58:55.095619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098248+0000","last_change":"2024-03-28T17:58:53.008301+0000","last_active":"2024-03-28T17:58:55.098248+0000","last_peered":"2024-03-28T17:58:55.098248+0000","last_clean":"2024-03-28T17:58:55.098248+0000","last_became_active":"2024-03-28T17:58:53.007920+0000","last_became_peered":"2024-03-28T17:58:53.007920+0000","last_unstale":"2024-03-28T17:58:55.098248+0000","last_undegraded":"2024-03-28T17:58:55.098248+0000","last_fullsized":"2024-03-28T17:58:55.098248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T17:58:58.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T17:58:58.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.726+0000 7f968be00640 1 Processor -- start 2024-03-28T17:58:58.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.727+0000 7f968be00640 1 -- start start 2024-03-28T17:58:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968be00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 0x7f968c0b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f968c0b7ec0 0x7f968c0b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968be00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0b6e60 con 0x7f968c15dd60 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c0b6c30 con 0x7f968c0b7ec0 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968be00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c0b6090 con 0x7f968c0b56f0 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f968c0b7ec0 0x7f968c0b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f968c0b7ec0 0x7f968c0b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46776/0 (socket says 172.21.15.62:46776) 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.728+0000 7f968b400640 1 -- 172.21.15.62:0/4138092809 learned_addr learned my addr 172.21.15.62:0/4138092809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968a000640 1 --2- 172.21.15.62:0/4138092809 >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 0x7f968c0b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968b400640 1 -- 172.21.15.62:0/4138092809 >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 msgr2=0x7f968c0b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968b400640 1 --2- 172.21.15.62:0/4138092809 >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 0x7f968c0b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968b400640 1 -- 172.21.15.62:0/4138092809 >> v1:172.21.15.81:6789/0 conn(0x7f968c15dd60 legacy=0x7f968c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:58.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968b400640 1 -- 172.21.15.62:0/4138092809 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0b7100 con 0x7f968c0b7ec0 2024-03-28T17:58:58.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.729+0000 7f968a000640 1 --2- 172.21.15.62:0/4138092809 >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 0x7f968c0b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.730+0000 7f968b400640 1 --2- 172.21.15.62:0/4138092809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b7ec0 0x7f968c0b82f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f968c074f20 tx=0x7f968402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ee8a1c09a3a8f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.730+0000 7f9689600640 1 -- 172.21.15.62:0/4138092809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684034070 con 0x7f968c0b7ec0 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.730+0000 7f9689600640 1 -- 172.21.15.62:0/4138092809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684030860 con 0x7f968c0b7ec0 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.730+0000 7f9689600640 1 -- 172.21.15.62:0/4138092809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684030c30 con 0x7f968c0b7ec0 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 -- 172.21.15.62:0/4138092809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b7ec0 msgr2=0x7f968c0b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 --2- 172.21.15.62:0/4138092809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b7ec0 0x7f968c0b82f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f968c074f20 tx=0x7f968402b9a0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 -- 172.21.15.62:0/4138092809 shutdown_connections 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 --2- 172.21.15.62:0/4138092809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b7ec0 0x7f968c0b82f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 --2- 172.21.15.62:0/4138092809 >> v2:172.21.15.81:3300/0 conn(0x7f968c0b56f0 0x7f968c0b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 -- 172.21.15.62:0/4138092809 >> 172.21.15.62:0/4138092809 conn(0x7f968c071ae0 msgr2=0x7f968c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 -- 172.21.15.62:0/4138092809 shutdown_connections 2024-03-28T17:58:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.731+0000 7f968be00640 1 -- 172.21.15.62:0/4138092809 wait complete. 2024-03-28T17:58:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.733+0000 7f968be00640 1 Processor -- start 2024-03-28T17:58:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.733+0000 7f968be00640 1 -- start start 2024-03-28T17:58:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f968c0b7ec0 0x7f968c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 0x7f968c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0b79a0 con 0x7f968c0b56f0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0b76c0 con 0x7f968c15dd60 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968be00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0b9bc0 con 0x7f968c0b7ec0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46782/0 (socket says 172.21.15.62:46782) 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 -- 172.21.15.62:0/274255012 learned_addr learned my addr 172.21.15.62:0/274255012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968a000640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f968c0b7ec0 0x7f968c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968b400640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 0x7f968c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 -- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f968c0b7ec0 msgr2=0x7f968c15e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f968c0b7ec0 0x7f968c15e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 -- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 msgr2=0x7f968c0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 0x7f968c0cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968aa00640 1 -- 172.21.15.62:0/274255012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c15e680 con 0x7f968c0b56f0 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.734+0000 7f968b400640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 0x7f968c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f968aa00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f967c00b3f0 tx=0x7f967c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c015020 con 0x7f968c0b56f0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c011830 con 0x7f968c0b56f0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c011c20 con 0x7f968c0b56f0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c0cb3d0 con 0x7f968c0b56f0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.735+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c14def0 con 0x7f968c0b56f0 2024-03-28T17:58:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.736+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c072b90 con 0x7f968c0b56f0 2024-03-28T17:58:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.737+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f967c018020 con 0x7f968c0b56f0 2024-03-28T17:58:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.738+0000 7f9682a00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 0x7f966806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.738+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f967c08aab0 con 0x7f968c0b56f0 2024-03-28T17:58:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.738+0000 7f968a000640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 0x7f966806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.739+0000 7f968a000640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 0x7f966806e0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9674003fd0 tx=0x7f96740039f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.742+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f967c01f020 con 0x7f968c0b56f0 2024-03-28T17:58:58.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.930+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f968c04f4e0 con 0x7f966806bc00 2024-03-28T17:58:58.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.950+0000 7f9682a00640 1 -- 172.21.15.62:0/274255012 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f968c04f4e0 con 0x7f966806bc00 2024-03-28T17:58:58.953 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:58.955 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.954+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 msgr2=0x7f966806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.954+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 0x7f966806e0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9674003fd0 tx=0x7f96740039f0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 msgr2=0x7f968c0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f967c00b3f0 tx=0x7f967c00b8b0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 shutdown_connections 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f966806bc00 0x7f966806e0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f968c15dd60 0x7f968c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f968c0b7ec0 0x7f968c15e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 --2- 172.21.15.62:0/274255012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c0b56f0 0x7f968c0cd5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 >> 172.21.15.62:0/274255012 conn(0x7f968c071ae0 msgr2=0x7f968c0b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 shutdown_connections 2024-03-28T17:58:58.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:58.955+0000 7f968be00640 1 -- 172.21.15.62:0/274255012 wait complete. 2024-03-28T17:58:58.972 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-28T17:58:58.305255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001666"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096023+0000","last_change":"2024-03-28T17:58:54.011100+0000","last_active":"2024-03-28T17:58:55.096023+0000","last_peered":"2024-03-28T17:58:55.096023+0000","last_clean":"2024-03-28T17:58:55.096023+0000","last_became_active":"2024-03-28T17:58:54.010769+0000","last_became_peered":"2024-03-28T17:58:54.010769+0000","last_unstale":"2024-03-28T17:58:55.096023+0000","last_undegraded":"2024-03-28T17:58:55.096023+0000","last_fullsized":"2024-03-28T17:58:55.096023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999288+0000","last_change":"2024-03-28T17:58:54.016056+0000","last_active":"2024-03-28T17:58:54.999288+0000","last_peered":"2024-03-28T17:58:54.999288+0000","last_clean":"2024-03-28T17:58:54.999288+0000","last_became_active":"2024-03-28T17:58:54.012351+0000","last_became_peered":"2024-03-28T17:58:54.012351+0000","last_unstale":"2024-03-28T17:58:54.999288+0000","last_undegraded":"2024-03-28T17:58:54.999288+0000","last_fullsized":"2024-03-28T17:58:54.999288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097234+0000","last_change":"2024-03-28T17:58:53.996524+0000","last_active":"2024-03-28T17:58:55.097234+0000","last_peered":"2024-03-28T17:58:55.097234+0000","last_clean":"2024-03-28T17:58:55.097234+0000","last_became_active":"2024-03-28T17:58:53.996197+0000","last_became_peered":"2024-03-28T17:58:53.996197+0000","last_unstale":"2024-03-28T17:58:55.097234+0000","last_undegraded":"2024-03-28T17:58:55.097234+0000","last_fullsized":"2024-03-28T17:58:55.097234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:16.038136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.001103+0000","last_change":"2024-03-28T17:58:54.009958+0000","last_active":"2024-03-28T17:58:55.001103+0000","last_peered":"2024-03-28T17:58:55.001103+0000","last_clean":"2024-03-28T17:58:55.001103+0000","last_became_active":"2024-03-28T17:58:54.009279+0000","last_became_peered":"2024-03-28T17:58:54.009279+0000","last_unstale":"2024-03-28T17:58:55.001103+0000","last_undegraded":"2024-03-28T17:58:55.001103+0000","last_fullsized":"2024-03-28T17:58:55.001103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095127+0000","last_change":"2024-03-28T17:58:54.007553+0000","last_active":"2024-03-28T17:58:55.095127+0000","last_peered":"2024-03-28T17:58:55.095127+0000","last_clean":"2024-03-28T17:58:55.095127+0000","last_became_active":"2024-03-28T17:58:54.007177+0000","last_became_peered":"2024-03-28T17:58:54.007177+0000","last_unstale":"2024-03-28T17:58:55.095127+0000","last_undegraded":"2024-03-28T17:58:55.095127+0000","last_fullsized":"2024-03-28T17:58:55.095127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999258+0000","last_change":"2024-03-28T17:58:53.998917+0000","last_active":"2024-03-28T17:58:54.999258+0000","last_peered":"2024-03-28T17:58:54.999258+0000","last_clean":"2024-03-28T17:58:54.999258+0000","last_became_active":"2024-03-28T17:58:53.998692+0000","last_became_peered":"2024-03-28T17:58:53.998692+0000","last_unstale":"2024-03-28T17:58:54.999258+0000","last_undegraded":"2024-03-28T17:58:54.999258+0000","last_fullsized":"2024-03-28T17:58:54.999258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094845+0000","last_change":"2024-03-28T17:58:54.007407+0000","last_active":"2024-03-28T17:58:55.094845+0000","last_peered":"2024-03-28T17:58:55.094845+0000","last_clean":"2024-03-28T17:58:55.094845+0000","last_became_active":"2024-03-28T17:58:54.007126+0000","last_became_peered":"2024-03-28T17:58:54.007126+0000","last_unstale":"2024-03-28T17:58:55.094845+0000","last_undegraded":"2024-03-28T17:58:55.094845+0000","last_fullsized":"2024-03-28T17:58:55.094845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094632+0000","last_change":"2024-03-28T17:58:54.021995+0000","last_active":"2024-03-28T17:58:55.094632+0000","last_peered":"2024-03-28T17:58:55.094632+0000","last_clean":"2024-03-28T17:58:55.094632+0000","last_became_active":"2024-03-28T17:58:54.021661+0000","last_became_peered":"2024-03-28T17:58:54.021661+0000","last_unstale":"2024-03-28T17:58:55.094632+0000","last_undegraded":"2024-03-28T17:58:55.094632+0000","last_fullsized":"2024-03-28T17:58:55.094632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096407+0000","last_change":"2024-03-28T17:58:54.014331+0000","last_active":"2024-03-28T17:58:55.096407+0000","last_peered":"2024-03-28T17:58:55.096407+0000","last_clean":"2024-03-28T17:58:55.096407+0000","last_became_active":"2024-03-28T17:58:54.014014+0000","last_became_peered":"2024-03-28T17:58:54.014014+0000","last_unstale":"2024-03-28T17:58:55.096407+0000","last_undegraded":"2024-03-28T17:58:55.096407+0000","last_fullsized":"2024-03-28T17:58:55.096407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019925+0000","last_change":"2024-03-28T17:58:54.013693+0000","last_active":"2024-03-28T17:58:55.019925+0000","last_peered":"2024-03-28T17:58:55.019925+0000","last_clean":"2024-03-28T17:58:55.019925+0000","last_became_active":"2024-03-28T17:58:54.013397+0000","last_became_peered":"2024-03-28T17:58:54.013397+0000","last_unstale":"2024-03-28T17:58:55.019925+0000","last_undegraded":"2024-03-28T17:58:55.019925+0000","last_fullsized":"2024-03-28T17:58:55.019925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019799+0000","last_change":"2024-03-28T17:58:54.014516+0000","last_active":"2024-03-28T17:58:55.019799+0000","last_peered":"2024-03-28T17:58:55.019799+0000","last_clean":"2024-03-28T17:58:55.019799+0000","last_became_active":"2024-03-28T17:58:54.014122+0000","last_became_peered":"2024-03-28T17:58:54.014122+0000","last_unstale":"2024-03-28T17:58:55.019799+0000","last_undegraded":"2024-03-28T17:58:55.019799+0000","last_fullsized":"2024-03-28T17:58:55.019799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097339+0000","last_change":"2024-03-28T17:58:54.015180+0000","last_active":"2024-03-28T17:58:55.097339+0000","last_peered":"2024-03-28T17:58:55.097339+0000","last_clean":"2024-03-28T17:58:55.097339+0000","last_became_active":"2024-03-28T17:58:54.008685+0000","last_became_peered":"2024-03-28T17:58:54.008685+0000","last_unstale":"2024-03-28T17:58:55.097339+0000","last_undegraded":"2024-03-28T17:58:55.097339+0000","last_fullsized":"2024-03-28T17:58:55.097339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095295+0000","last_change":"2024-03-28T17:58:54.012021+0000","last_active":"2024-03-28T17:58:55.095295+0000","last_peered":"2024-03-28T17:58:55.095295+0000","last_clean":"2024-03-28T17:58:55.095295+0000","last_became_active":"2024-03-28T17:58:54.011650+0000","last_became_peered":"2024-03-28T17:58:54.011650+0000","last_unstale":"2024-03-28T17:58:55.095295+0000","last_undegraded":"2024-03-28T17:58:55.095295+0000","last_fullsized":"2024-03-28T17:58:55.095295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097361+0000","last_change":"2024-03-28T17:58:54.011097+0000","last_active":"2024-03-28T17:58:55.097361+0000","last_peered":"2024-03-28T17:58:55.097361+0000","last_clean":"2024-03-28T17:58:55.097361+0000","last_became_active":"2024-03-28T17:58:54.010790+0000","last_became_peered":"2024-03-28T17:58:54.010790+0000","last_unstale":"2024-03-28T17:58:55.097361+0000","last_undegraded":"2024-03-28T17:58:55.097361+0000","last_fullsized":"2024-03-28T17:58:55.097361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:51:58.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996815+0000","last_change":"2024-03-28T17:58:54.022714+0000","last_active":"2024-03-28T17:58:54.996815+0000","last_peered":"2024-03-28T17:58:54.996815+0000","last_clean":"2024-03-28T17:58:54.996815+0000","last_became_active":"2024-03-28T17:58:54.022409+0000","last_became_peered":"2024-03-28T17:58:54.022409+0000","last_unstale":"2024-03-28T17:58:54.996815+0000","last_undegraded":"2024-03-28T17:58:54.996815+0000","last_fullsized":"2024-03-28T17:58:54.996815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999073+0000","last_change":"2024-03-28T17:58:54.005916+0000","last_active":"2024-03-28T17:58:54.999073+0000","last_peered":"2024-03-28T17:58:54.999073+0000","last_clean":"2024-03-28T17:58:54.999073+0000","last_became_active":"2024-03-28T17:58:54.005631+0000","last_became_peered":"2024-03-28T17:58:54.005631+0000","last_unstale":"2024-03-28T17:58:54.999073+0000","last_undegraded":"2024-03-28T17:58:54.999073+0000","last_fullsized":"2024-03-28T17:58:54.999073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000894+0000","last_change":"2024-03-28T17:58:54.005717+0000","last_active":"2024-03-28T17:58:55.000894+0000","last_peered":"2024-03-28T17:58:55.000894+0000","last_clean":"2024-03-28T17:58:55.000894+0000","last_became_active":"2024-03-28T17:58:54.005397+0000","last_became_peered":"2024-03-28T17:58:54.005397+0000","last_unstale":"2024-03-28T17:58:55.000894+0000","last_undegraded":"2024-03-28T17:58:55.000894+0000","last_fullsized":"2024-03-28T17:58:55.000894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997043+0000","last_change":"2024-03-28T17:58:54.009673+0000","last_active":"2024-03-28T17:58:54.997043+0000","last_peered":"2024-03-28T17:58:54.997043+0000","last_clean":"2024-03-28T17:58:54.997043+0000","last_became_active":"2024-03-28T17:58:54.009387+0000","last_became_peered":"2024-03-28T17:58:54.009387+0000","last_unstale":"2024-03-28T17:58:54.997043+0000","last_undegraded":"2024-03-28T17:58:54.997043+0000","last_fullsized":"2024-03-28T17:58:54.997043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:52.016086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095794+0000","last_change":"2024-03-28T17:58:54.003074+0000","last_active":"2024-03-28T17:58:55.095794+0000","last_peered":"2024-03-28T17:58:55.095794+0000","last_clean":"2024-03-28T17:58:55.095794+0000","last_became_active":"2024-03-28T17:58:54.002854+0000","last_became_peered":"2024-03-28T17:58:54.002854+0000","last_unstale":"2024-03-28T17:58:55.095794+0000","last_undegraded":"2024-03-28T17:58:55.095794+0000","last_fullsized":"2024-03-28T17:58:55.095794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019743+0000","last_change":"2024-03-28T17:58:54.022729+0000","last_active":"2024-03-28T17:58:55.019743+0000","last_peered":"2024-03-28T17:58:55.019743+0000","last_clean":"2024-03-28T17:58:55.019743+0000","last_became_active":"2024-03-28T17:58:54.022377+0000","last_became_peered":"2024-03-28T17:58:54.022377+0000","last_unstale":"2024-03-28T17:58:55.019743+0000","last_undegraded":"2024-03-28T17:58:55.019743+0000","last_fullsized":"2024-03-28T17:58:55.019743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098073+0000","last_change":"2024-03-28T17:58:54.005642+0000","last_active":"2024-03-28T17:58:55.098073+0000","last_peered":"2024-03-28T17:58:55.098073+0000","last_clean":"2024-03-28T17:58:55.098073+0000","last_became_active":"2024-03-28T17:58:54.005344+0000","last_became_peered":"2024-03-28T17:58:54.005344+0000","last_unstale":"2024-03-28T17:58:55.098073+0000","last_undegraded":"2024-03-28T17:58:55.098073+0000","last_fullsized":"2024-03-28T17:58:55.098073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999309+0000","last_change":"2024-03-28T17:58:54.005142+0000","last_active":"2024-03-28T17:58:54.999309+0000","last_peered":"2024-03-28T17:58:54.999309+0000","last_clean":"2024-03-28T17:58:54.999309+0000","last_became_active":"2024-03-28T17:58:54.003593+0000","last_became_peered":"2024-03-28T17:58:54.003593+0000","last_unstale":"2024-03-28T17:58:54.999309+0000","last_undegraded":"2024-03-28T17:58:54.999309+0000","last_fullsized":"2024-03-28T17:58:54.999309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095680+0000","last_change":"2024-03-28T17:58:54.020402+0000","last_active":"2024-03-28T17:58:55.095680+0000","last_peered":"2024-03-28T17:58:55.095680+0000","last_clean":"2024-03-28T17:58:55.095680+0000","last_became_active":"2024-03-28T17:58:54.020055+0000","last_became_peered":"2024-03-28T17:58:54.020055+0000","last_unstale":"2024-03-28T17:58:55.095680+0000","last_undegraded":"2024-03-28T17:58:55.095680+0000","last_fullsized":"2024-03-28T17:58:55.095680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999282+0000","last_change":"2024-03-28T17:58:54.012462+0000","last_active":"2024-03-28T17:58:54.999282+0000","last_peered":"2024-03-28T17:58:54.999282+0000","last_clean":"2024-03-28T17:58:54.999282+0000","last_became_active":"2024-03-28T17:58:54.011568+0000","last_became_peered":"2024-03-28T17:58:54.011568+0000","last_unstale":"2024-03-28T17:58:54.999282+0000","last_undegraded":"2024-03-28T17:58:54.999282+0000","last_fullsized":"2024-03-28T17:58:54.999282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095340+0000","last_change":"2024-03-28T17:58:54.023228+0000","last_active":"2024-03-28T17:58:55.095340+0000","last_peered":"2024-03-28T17:58:55.095340+0000","last_clean":"2024-03-28T17:58:55.095340+0000","last_became_active":"2024-03-28T17:58:54.022910+0000","last_became_peered":"2024-03-28T17:58:54.022910+0000","last_unstale":"2024-03-28T17:58:55.095340+0000","last_undegraded":"2024-03-28T17:58:55.095340+0000","last_fullsized":"2024-03-28T17:58:55.095340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019938+0000","last_change":"2024-03-28T17:58:54.006053+0000","last_active":"2024-03-28T17:58:55.019938+0000","last_peered":"2024-03-28T17:58:55.019938+0000","last_clean":"2024-03-28T17:58:55.019938+0000","last_became_active":"2024-03-28T17:58:54.005693+0000","last_became_peered":"2024-03-28T17:58:54.005693+0000","last_unstale":"2024-03-28T17:58:55.019938+0000","last_undegraded":"2024-03-28T17:58:55.019938+0000","last_fullsized":"2024-03-28T17:58:55.019938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998856+0000","last_change":"2024-03-28T17:58:54.013535+0000","last_active":"2024-03-28T17:58:54.998856+0000","last_peered":"2024-03-28T17:58:54.998856+0000","last_clean":"2024-03-28T17:58:54.998856+0000","last_became_active":"2024-03-28T17:58:54.011843+0000","last_became_peered":"2024-03-28T17:58:54.011843+0000","last_unstale":"2024-03-28T17:58:54.998856+0000","last_undegraded":"2024-03-28T17:58:54.998856+0000","last_fullsized":"2024-03-28T17:58:54.998856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095519+0000","last_change":"2024-03-28T17:58:54.022600+0000","last_active":"2024-03-28T17:58:55.095519+0000","last_peered":"2024-03-28T17:58:55.095519+0000","last_clean":"2024-03-28T17:58:55.095519+0000","last_became_active":"2024-03-28T17:58:54.022354+0000","last_became_peered":"2024-03-28T17:58:54.022354+0000","last_unstale":"2024-03-28T17:58:55.095519+0000","last_undegraded":"2024-03-28T17:58:55.095519+0000","last_fullsized":"2024-03-28T17:58:55.095519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097778+0000","last_change":"2024-03-28T17:58:54.005723+0000","last_active":"2024-03-28T17:58:55.097778+0000","last_peered":"2024-03-28T17:58:55.097778+0000","last_clean":"2024-03-28T17:58:55.097778+0000","last_became_active":"2024-03-28T17:58:54.005383+0000","last_became_peered":"2024-03-28T17:58:54.005383+0000","last_unstale":"2024-03-28T17:58:55.097778+0000","last_undegraded":"2024-03-28T17:58:55.097778+0000","last_fullsized":"2024-03-28T17:58:55.097778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:54.012921+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:54.012471+0000","last_became_peered":"2024-03-28T17:58:54.012471+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996217+0000","last_change":"2024-03-28T17:58:53.996340+0000","last_active":"2024-03-28T17:58:54.996217+0000","last_peered":"2024-03-28T17:58:54.996217+0000","last_clean":"2024-03-28T17:58:54.996217+0000","last_became_active":"2024-03-28T17:58:53.995914+0000","last_became_peered":"2024-03-28T17:58:53.995914+0000","last_unstale":"2024-03-28T17:58:54.996217+0000","last_undegraded":"2024-03-28T17:58:54.996217+0000","last_fullsized":"2024-03-28T17:58:54.996217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019333+0000","last_change":"2024-03-28T17:58:54.012238+0000","last_active":"2024-03-28T17:58:55.019333+0000","last_peered":"2024-03-28T17:58:55.019333+0000","last_clean":"2024-03-28T17:58:55.019333+0000","last_became_active":"2024-03-28T17:58:54.011952+0000","last_became_peered":"2024-03-28T17:58:54.011952+0000","last_unstale":"2024-03-28T17:58:55.019333+0000","last_undegraded":"2024-03-28T17:58:55.019333+0000","last_fullsized":"2024-03-28T17:58:55.019333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096509+0000","last_change":"2024-03-28T17:58:54.011087+0000","last_active":"2024-03-28T17:58:55.096509+0000","last_peered":"2024-03-28T17:58:55.096509+0000","last_clean":"2024-03-28T17:58:55.096509+0000","last_became_active":"2024-03-28T17:58:54.010787+0000","last_became_peered":"2024-03-28T17:58:54.010787+0000","last_unstale":"2024-03-28T17:58:55.096509+0000","last_undegraded":"2024-03-28T17:58:55.096509+0000","last_fullsized":"2024-03-28T17:58:55.096509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:23.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095155+0000","last_change":"2024-03-28T17:58:53.002153+0000","last_active":"2024-03-28T17:58:55.095155+0000","last_peered":"2024-03-28T17:58:55.095155+0000","last_clean":"2024-03-28T17:58:55.095155+0000","last_became_active":"2024-03-28T17:58:53.001844+0000","last_became_peered":"2024-03-28T17:58:53.001844+0000","last_unstale":"2024-03-28T17:58:55.095155+0000","last_undegraded":"2024-03-28T17:58:55.095155+0000","last_fullsized":"2024-03-28T17:58:55.095155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:38:14.392145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097946+0000","last_change":"2024-03-28T17:58:54.027047+0000","last_active":"2024-03-28T17:58:55.097946+0000","last_peered":"2024-03-28T17:58:55.097946+0000","last_clean":"2024-03-28T17:58:55.097946+0000","last_became_active":"2024-03-28T17:58:54.022997+0000","last_became_peered":"2024-03-28T17:58:54.022997+0000","last_unstale":"2024-03-28T17:58:55.097946+0000","last_undegraded":"2024-03-28T17:58:55.097946+0000","last_fullsized":"2024-03-28T17:58:55.097946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019270+0000","last_change":"2024-03-28T17:58:53.000389+0000","last_active":"2024-03-28T17:58:55.019270+0000","last_peered":"2024-03-28T17:58:55.019270+0000","last_clean":"2024-03-28T17:58:55.019270+0000","last_became_active":"2024-03-28T17:58:53.000086+0000","last_became_peered":"2024-03-28T17:58:53.000086+0000","last_unstale":"2024-03-28T17:58:55.019270+0000","last_undegraded":"2024-03-28T17:58:55.019270+0000","last_fullsized":"2024-03-28T17:58:55.019270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094817+0000","last_change":"2024-03-28T17:58:53.004667+0000","last_active":"2024-03-28T17:58:55.094817+0000","last_peered":"2024-03-28T17:58:55.094817+0000","last_clean":"2024-03-28T17:58:55.094817+0000","last_became_active":"2024-03-28T17:58:53.004337+0000","last_became_peered":"2024-03-28T17:58:53.004337+0000","last_unstale":"2024-03-28T17:58:55.094817+0000","last_undegraded":"2024-03-28T17:58:55.094817+0000","last_fullsized":"2024-03-28T17:58:55.094817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094868+0000","last_change":"2024-03-28T17:58:54.008588+0000","last_active":"2024-03-28T17:58:55.094868+0000","last_peered":"2024-03-28T17:58:55.094868+0000","last_clean":"2024-03-28T17:58:55.094868+0000","last_became_active":"2024-03-28T17:58:54.005942+0000","last_became_peered":"2024-03-28T17:58:54.005942+0000","last_unstale":"2024-03-28T17:58:55.094868+0000","last_undegraded":"2024-03-28T17:58:55.094868+0000","last_fullsized":"2024-03-28T17:58:55.094868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999261+0000","last_change":"2024-03-28T17:58:53.003611+0000","last_active":"2024-03-28T17:58:54.999261+0000","last_peered":"2024-03-28T17:58:54.999261+0000","last_clean":"2024-03-28T17:58:54.999261+0000","last_became_active":"2024-03-28T17:58:53.003272+0000","last_became_peered":"2024-03-28T17:58:53.003272+0000","last_unstale":"2024-03-28T17:58:54.999261+0000","last_undegraded":"2024-03-28T17:58:54.999261+0000","last_fullsized":"2024-03-28T17:58:54.999261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996592+0000","last_change":"2024-03-28T17:58:54.013586+0000","last_active":"2024-03-28T17:58:54.996592+0000","last_peered":"2024-03-28T17:58:54.996592+0000","last_clean":"2024-03-28T17:58:54.996592+0000","last_became_active":"2024-03-28T17:58:54.013293+0000","last_became_peered":"2024-03-28T17:58:54.013293+0000","last_unstale":"2024-03-28T17:58:54.996592+0000","last_undegraded":"2024-03-28T17:58:54.996592+0000","last_fullsized":"2024-03-28T17:58:54.996592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019821+0000","last_change":"2024-03-28T17:58:53.005607+0000","last_active":"2024-03-28T17:58:55.019821+0000","last_peered":"2024-03-28T17:58:55.019821+0000","last_clean":"2024-03-28T17:58:55.019821+0000","last_became_active":"2024-03-28T17:58:53.005306+0000","last_became_peered":"2024-03-28T17:58:53.005306+0000","last_unstale":"2024-03-28T17:58:55.019821+0000","last_undegraded":"2024-03-28T17:58:55.019821+0000","last_fullsized":"2024-03-28T17:58:55.019821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000404+0000","last_change":"2024-03-28T17:58:54.006871+0000","last_active":"2024-03-28T17:58:55.000404+0000","last_peered":"2024-03-28T17:58:55.000404+0000","last_clean":"2024-03-28T17:58:55.000404+0000","last_became_active":"2024-03-28T17:58:54.006521+0000","last_became_peered":"2024-03-28T17:58:54.006521+0000","last_unstale":"2024-03-28T17:58:55.000404+0000","last_undegraded":"2024-03-28T17:58:55.000404+0000","last_fullsized":"2024-03-28T17:58:55.000404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000147+0000","last_change":"2024-03-28T17:58:53.008708+0000","last_active":"2024-03-28T17:58:55.000147+0000","last_peered":"2024-03-28T17:58:55.000147+0000","last_clean":"2024-03-28T17:58:55.000147+0000","last_became_active":"2024-03-28T17:58:53.008424+0000","last_became_peered":"2024-03-28T17:58:53.008424+0000","last_unstale":"2024-03-28T17:58:55.000147+0000","last_undegraded":"2024-03-28T17:58:55.000147+0000","last_fullsized":"2024-03-28T17:58:55.000147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098103+0000","last_change":"2024-03-28T17:58:54.007645+0000","last_active":"2024-03-28T17:58:55.098103+0000","last_peered":"2024-03-28T17:58:55.098103+0000","last_clean":"2024-03-28T17:58:55.098103+0000","last_became_active":"2024-03-28T17:58:54.007359+0000","last_became_peered":"2024-03-28T17:58:54.007359+0000","last_unstale":"2024-03-28T17:58:55.098103+0000","last_undegraded":"2024-03-28T17:58:55.098103+0000","last_fullsized":"2024-03-28T17:58:55.098103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998573+0000","last_change":"2024-03-28T17:58:53.000532+0000","last_active":"2024-03-28T17:58:54.998573+0000","last_peered":"2024-03-28T17:58:54.998573+0000","last_clean":"2024-03-28T17:58:54.998573+0000","last_became_active":"2024-03-28T17:58:53.000238+0000","last_became_peered":"2024-03-28T17:58:53.000238+0000","last_unstale":"2024-03-28T17:58:54.998573+0000","last_undegraded":"2024-03-28T17:58:54.998573+0000","last_fullsized":"2024-03-28T17:58:54.998573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097269+0000","last_change":"2024-03-28T17:58:54.011919+0000","last_active":"2024-03-28T17:58:55.097269+0000","last_peered":"2024-03-28T17:58:55.097269+0000","last_clean":"2024-03-28T17:58:55.097269+0000","last_became_active":"2024-03-28T17:58:54.008256+0000","last_became_peered":"2024-03-28T17:58:54.008256+0000","last_unstale":"2024-03-28T17:58:55.097269+0000","last_undegraded":"2024-03-28T17:58:55.097269+0000","last_fullsized":"2024-03-28T17:58:55.097269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:16.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019656+0000","last_change":"2024-03-28T17:58:53.004395+0000","last_active":"2024-03-28T17:58:55.019656+0000","last_peered":"2024-03-28T17:58:55.019656+0000","last_clean":"2024-03-28T17:58:55.019656+0000","last_became_active":"2024-03-28T17:58:52.999296+0000","last_became_peered":"2024-03-28T17:58:52.999296+0000","last_unstale":"2024-03-28T17:58:55.019656+0000","last_undegraded":"2024-03-28T17:58:55.019656+0000","last_fullsized":"2024-03-28T17:58:55.019656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095473+0000","last_change":"2024-03-28T17:58:54.004875+0000","last_active":"2024-03-28T17:58:55.095473+0000","last_peered":"2024-03-28T17:58:55.095473+0000","last_clean":"2024-03-28T17:58:55.095473+0000","last_became_active":"2024-03-28T17:58:54.004597+0000","last_became_peered":"2024-03-28T17:58:54.004597+0000","last_unstale":"2024-03-28T17:58:55.095473+0000","last_undegraded":"2024-03-28T17:58:55.095473+0000","last_fullsized":"2024-03-28T17:58:55.095473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098023+0000","last_change":"2024-03-28T17:58:53.009461+0000","last_active":"2024-03-28T17:58:55.098023+0000","last_peered":"2024-03-28T17:58:55.098023+0000","last_clean":"2024-03-28T17:58:55.098023+0000","last_became_active":"2024-03-28T17:58:53.008867+0000","last_became_peered":"2024-03-28T17:58:53.008867+0000","last_unstale":"2024-03-28T17:58:55.098023+0000","last_undegraded":"2024-03-28T17:58:55.098023+0000","last_fullsized":"2024-03-28T17:58:55.098023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:57:02.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098575+0000","last_change":"2024-03-28T17:58:54.001368+0000","last_active":"2024-03-28T17:58:55.098575+0000","last_peered":"2024-03-28T17:58:55.098575+0000","last_clean":"2024-03-28T17:58:55.098575+0000","last_became_active":"2024-03-28T17:58:54.000744+0000","last_became_peered":"2024-03-28T17:58:54.000744+0000","last_unstale":"2024-03-28T17:58:55.098575+0000","last_undegraded":"2024-03-28T17:58:55.098575+0000","last_fullsized":"2024-03-28T17:58:55.098575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999461+0000","last_change":"2024-03-28T17:58:53.007374+0000","last_active":"2024-03-28T17:58:54.999461+0000","last_peered":"2024-03-28T17:58:54.999461+0000","last_clean":"2024-03-28T17:58:54.999461+0000","last_became_active":"2024-03-28T17:58:53.005001+0000","last_became_peered":"2024-03-28T17:58:53.005001+0000","last_unstale":"2024-03-28T17:58:54.999461+0000","last_undegraded":"2024-03-28T17:58:54.999461+0000","last_fullsized":"2024-03-28T17:58:54.999461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095283+0000","last_change":"2024-03-28T17:58:54.020789+0000","last_active":"2024-03-28T17:58:55.095283+0000","last_peered":"2024-03-28T17:58:55.095283+0000","last_clean":"2024-03-28T17:58:55.095283+0000","last_became_active":"2024-03-28T17:58:54.020338+0000","last_became_peered":"2024-03-28T17:58:54.020338+0000","last_unstale":"2024-03-28T17:58:55.095283+0000","last_undegraded":"2024-03-28T17:58:55.095283+0000","last_fullsized":"2024-03-28T17:58:55.095283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019218+0000","last_change":"2024-03-28T17:58:53.001274+0000","last_active":"2024-03-28T17:58:55.019218+0000","last_peered":"2024-03-28T17:58:55.019218+0000","last_clean":"2024-03-28T17:58:55.019218+0000","last_became_active":"2024-03-28T17:58:53.000949+0000","last_became_peered":"2024-03-28T17:58:53.000949+0000","last_unstale":"2024-03-28T17:58:55.019218+0000","last_undegraded":"2024-03-28T17:58:55.019218+0000","last_fullsized":"2024-03-28T17:58:55.019218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019704+0000","last_change":"2024-03-28T17:58:54.009630+0000","last_active":"2024-03-28T17:58:55.019704+0000","last_peered":"2024-03-28T17:58:55.019704+0000","last_clean":"2024-03-28T17:58:55.019704+0000","last_became_active":"2024-03-28T17:58:54.009072+0000","last_became_peered":"2024-03-28T17:58:54.009072+0000","last_unstale":"2024-03-28T17:58:55.019704+0000","last_undegraded":"2024-03-28T17:58:55.019704+0000","last_fullsized":"2024-03-28T17:58:55.019704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.995993+0000","last_change":"2024-03-28T17:58:53.002389+0000","last_active":"2024-03-28T17:58:54.995993+0000","last_peered":"2024-03-28T17:58:54.995993+0000","last_clean":"2024-03-28T17:58:54.995993+0000","last_became_active":"2024-03-28T17:58:53.001859+0000","last_became_peered":"2024-03-28T17:58:53.001859+0000","last_unstale":"2024-03-28T17:58:54.995993+0000","last_undegraded":"2024-03-28T17:58:54.995993+0000","last_fullsized":"2024-03-28T17:58:54.995993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095130+0000","last_change":"2024-03-28T17:58:54.022077+0000","last_active":"2024-03-28T17:58:55.095130+0000","last_peered":"2024-03-28T17:58:55.095130+0000","last_clean":"2024-03-28T17:58:55.095130+0000","last_became_active":"2024-03-28T17:58:54.021287+0000","last_became_peered":"2024-03-28T17:58:54.021287+0000","last_unstale":"2024-03-28T17:58:55.095130+0000","last_undegraded":"2024-03-28T17:58:55.095130+0000","last_fullsized":"2024-03-28T17:58:55.095130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097510+0000","last_change":"2024-03-28T17:58:53.002001+0000","last_active":"2024-03-28T17:58:55.097510+0000","last_peered":"2024-03-28T17:58:55.097510+0000","last_clean":"2024-03-28T17:58:55.097510+0000","last_became_active":"2024-03-28T17:58:53.001734+0000","last_became_peered":"2024-03-28T17:58:53.001734+0000","last_unstale":"2024-03-28T17:58:55.097510+0000","last_undegraded":"2024-03-28T17:58:55.097510+0000","last_fullsized":"2024-03-28T17:58:55.097510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:13:29.578775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095671+0000","last_change":"2024-03-28T17:58:54.021625+0000","last_active":"2024-03-28T17:58:55.095671+0000","last_peered":"2024-03-28T17:58:55.095671+0000","last_clean":"2024-03-28T17:58:55.095671+0000","last_became_active":"2024-03-28T17:58:54.020370+0000","last_became_peered":"2024-03-28T17:58:54.020370+0000","last_unstale":"2024-03-28T17:58:55.095671+0000","last_undegraded":"2024-03-28T17:58:55.095671+0000","last_fullsized":"2024-03-28T17:58:55.095671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997096+0000","last_change":"2024-03-28T17:58:52.997706+0000","last_active":"2024-03-28T17:58:54.997096+0000","last_peered":"2024-03-28T17:58:54.997096+0000","last_clean":"2024-03-28T17:58:54.997096+0000","last_became_active":"2024-03-28T17:58:52.997002+0000","last_became_peered":"2024-03-28T17:58:52.997002+0000","last_unstale":"2024-03-28T17:58:54.997096+0000","last_undegraded":"2024-03-28T17:58:54.997096+0000","last_fullsized":"2024-03-28T17:58:54.997096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096799+0000","last_change":"2024-03-28T17:58:54.021721+0000","last_active":"2024-03-28T17:58:55.096799+0000","last_peered":"2024-03-28T17:58:55.096799+0000","last_clean":"2024-03-28T17:58:55.096799+0000","last_became_active":"2024-03-28T17:58:54.020973+0000","last_became_peered":"2024-03-28T17:58:54.020973+0000","last_unstale":"2024-03-28T17:58:55.096799+0000","last_undegraded":"2024-03-28T17:58:55.096799+0000","last_fullsized":"2024-03-28T17:58:55.096799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:29:06.711377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998981+0000","last_change":"2024-03-28T17:58:53.007402+0000","last_active":"2024-03-28T17:58:54.998981+0000","last_peered":"2024-03-28T17:58:54.998981+0000","last_clean":"2024-03-28T17:58:54.998981+0000","last_became_active":"2024-03-28T17:58:53.007131+0000","last_became_peered":"2024-03-28T17:58:53.007131+0000","last_unstale":"2024-03-28T17:58:54.998981+0000","last_undegraded":"2024-03-28T17:58:54.998981+0000","last_fullsized":"2024-03-28T17:58:54.998981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000866+0000","last_change":"2024-03-28T17:58:54.009567+0000","last_active":"2024-03-28T17:58:55.000866+0000","last_peered":"2024-03-28T17:58:55.000866+0000","last_clean":"2024-03-28T17:58:55.000866+0000","last_became_active":"2024-03-28T17:58:54.008926+0000","last_became_peered":"2024-03-28T17:58:54.008926+0000","last_unstale":"2024-03-28T17:58:55.000866+0000","last_undegraded":"2024-03-28T17:58:55.000866+0000","last_fullsized":"2024-03-28T17:58:55.000866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996703+0000","last_change":"2024-03-28T17:58:52.995453+0000","last_active":"2024-03-28T17:58:54.996703+0000","last_peered":"2024-03-28T17:58:54.996703+0000","last_clean":"2024-03-28T17:58:54.996703+0000","last_became_active":"2024-03-28T17:58:52.995177+0000","last_became_peered":"2024-03-28T17:58:52.995177+0000","last_unstale":"2024-03-28T17:58:54.996703+0000","last_undegraded":"2024-03-28T17:58:54.996703+0000","last_fullsized":"2024-03-28T17:58:54.996703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095614+0000","last_change":"2024-03-28T17:58:54.004742+0000","last_active":"2024-03-28T17:58:55.095614+0000","last_peered":"2024-03-28T17:58:55.095614+0000","last_clean":"2024-03-28T17:58:55.095614+0000","last_became_active":"2024-03-28T17:58:54.004434+0000","last_became_peered":"2024-03-28T17:58:54.004434+0000","last_unstale":"2024-03-28T17:58:55.095614+0000","last_undegraded":"2024-03-28T17:58:55.095614+0000","last_fullsized":"2024-03-28T17:58:55.095614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094833+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T17:58:55.094833+0000","last_peered":"2024-03-28T17:58:55.094833+0000","last_clean":"2024-03-28T17:58:55.094833+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T17:58:55.094833+0000","last_undegraded":"2024-03-28T17:58:55.094833+0000","last_fullsized":"2024-03-28T17:58:55.094833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999042+0000","last_change":"2024-03-28T17:58:52.995644+0000","last_active":"2024-03-28T17:58:54.999042+0000","last_peered":"2024-03-28T17:58:54.999042+0000","last_clean":"2024-03-28T17:58:54.999042+0000","last_became_active":"2024-03-28T17:58:52.995196+0000","last_became_peered":"2024-03-28T17:58:52.995196+0000","last_unstale":"2024-03-28T17:58:54.999042+0000","last_undegraded":"2024-03-28T17:58:54.999042+0000","last_fullsized":"2024-03-28T17:58:54.999042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095066+0000","last_change":"2024-03-28T17:58:54.015875+0000","last_active":"2024-03-28T17:58:55.095066+0000","last_peered":"2024-03-28T17:58:55.095066+0000","last_clean":"2024-03-28T17:58:55.095066+0000","last_became_active":"2024-03-28T17:58:54.014187+0000","last_became_peered":"2024-03-28T17:58:54.014187+0000","last_unstale":"2024-03-28T17:58:55.095066+0000","last_undegraded":"2024-03-28T17:58:55.095066+0000","last_fullsized":"2024-03-28T17:58:55.095066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094699+0000","last_change":"2024-03-28T17:58:53.010263+0000","last_active":"2024-03-28T17:58:55.094699+0000","last_peered":"2024-03-28T17:58:55.094699+0000","last_clean":"2024-03-28T17:58:55.094699+0000","last_became_active":"2024-03-28T17:58:53.009990+0000","last_became_peered":"2024-03-28T17:58:53.009990+0000","last_unstale":"2024-03-28T17:58:55.094699+0000","last_undegraded":"2024-03-28T17:58:55.094699+0000","last_fullsized":"2024-03-28T17:58:55.094699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097226+0000","last_change":"2024-03-28T17:58:54.022349+0000","last_active":"2024-03-28T17:58:55.097226+0000","last_peered":"2024-03-28T17:58:55.097226+0000","last_clean":"2024-03-28T17:58:55.097226+0000","last_became_active":"2024-03-28T17:58:54.021997+0000","last_became_peered":"2024-03-28T17:58:54.021997+0000","last_unstale":"2024-03-28T17:58:55.097226+0000","last_undegraded":"2024-03-28T17:58:55.097226+0000","last_fullsized":"2024-03-28T17:58:55.097226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:20.314101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996301+0000","last_change":"2024-03-28T17:58:52.994588+0000","last_active":"2024-03-28T17:58:54.996301+0000","last_peered":"2024-03-28T17:58:54.996301+0000","last_clean":"2024-03-28T17:58:54.996301+0000","last_became_active":"2024-03-28T17:58:52.994338+0000","last_became_peered":"2024-03-28T17:58:52.994338+0000","last_unstale":"2024-03-28T17:58:54.996301+0000","last_undegraded":"2024-03-28T17:58:54.996301+0000","last_fullsized":"2024-03-28T17:58:54.996301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997461+0000","last_change":"2024-03-28T17:58:53.998446+0000","last_active":"2024-03-28T17:58:54.997461+0000","last_peered":"2024-03-28T17:58:54.997461+0000","last_clean":"2024-03-28T17:58:54.997461+0000","last_became_active":"2024-03-28T17:58:53.998163+0000","last_became_peered":"2024-03-28T17:58:53.998163+0000","last_unstale":"2024-03-28T17:58:54.997461+0000","last_undegraded":"2024-03-28T17:58:54.997461+0000","last_fullsized":"2024-03-28T17:58:54.997461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:52.998174+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:52.997265+0000","last_became_peered":"2024-03-28T17:58:52.997265+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096877+0000","last_change":"2024-03-28T17:58:54.017676+0000","last_active":"2024-03-28T17:58:55.096877+0000","last_peered":"2024-03-28T17:58:55.096877+0000","last_clean":"2024-03-28T17:58:55.096877+0000","last_became_active":"2024-03-28T17:58:54.016900+0000","last_became_peered":"2024-03-28T17:58:54.016900+0000","last_unstale":"2024-03-28T17:58:55.096877+0000","last_undegraded":"2024-03-28T17:58:55.096877+0000","last_fullsized":"2024-03-28T17:58:55.096877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:45.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998525+0000","last_change":"2024-03-28T17:58:52.997517+0000","last_active":"2024-03-28T17:58:54.998525+0000","last_peered":"2024-03-28T17:58:54.998525+0000","last_clean":"2024-03-28T17:58:54.998525+0000","last_became_active":"2024-03-28T17:58:52.997207+0000","last_became_peered":"2024-03-28T17:58:52.997207+0000","last_unstale":"2024-03-28T17:58:54.998525+0000","last_undegraded":"2024-03-28T17:58:54.998525+0000","last_fullsized":"2024-03-28T17:58:54.998525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999951+0000","last_change":"2024-03-28T17:58:54.011090+0000","last_active":"2024-03-28T17:58:54.999951+0000","last_peered":"2024-03-28T17:58:54.999951+0000","last_clean":"2024-03-28T17:58:54.999951+0000","last_became_active":"2024-03-28T17:58:54.009864+0000","last_became_peered":"2024-03-28T17:58:54.009864+0000","last_unstale":"2024-03-28T17:58:54.999951+0000","last_undegraded":"2024-03-28T17:58:54.999951+0000","last_fullsized":"2024-03-28T17:58:54.999951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095825+0000","last_change":"2024-03-28T17:58:53.011734+0000","last_active":"2024-03-28T17:58:55.095825+0000","last_peered":"2024-03-28T17:58:55.095825+0000","last_clean":"2024-03-28T17:58:55.095825+0000","last_became_active":"2024-03-28T17:58:53.011209+0000","last_became_peered":"2024-03-28T17:58:53.011209+0000","last_unstale":"2024-03-28T17:58:55.095825+0000","last_undegraded":"2024-03-28T17:58:55.095825+0000","last_fullsized":"2024-03-28T17:58:55.095825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997836+0000","last_change":"2024-03-28T17:58:54.015424+0000","last_active":"2024-03-28T17:58:54.997836+0000","last_peered":"2024-03-28T17:58:54.997836+0000","last_clean":"2024-03-28T17:58:54.997836+0000","last_became_active":"2024-03-28T17:58:54.011122+0000","last_became_peered":"2024-03-28T17:58:54.011122+0000","last_unstale":"2024-03-28T17:58:54.997836+0000","last_undegraded":"2024-03-28T17:58:54.997836+0000","last_fullsized":"2024-03-28T17:58:54.997836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999477+0000","last_change":"2024-03-28T17:58:53.008757+0000","last_active":"2024-03-28T17:58:54.999477+0000","last_peered":"2024-03-28T17:58:54.999477+0000","last_clean":"2024-03-28T17:58:54.999477+0000","last_became_active":"2024-03-28T17:58:53.008455+0000","last_became_peered":"2024-03-28T17:58:53.008455+0000","last_unstale":"2024-03-28T17:58:54.999477+0000","last_undegraded":"2024-03-28T17:58:54.999477+0000","last_fullsized":"2024-03-28T17:58:54.999477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096368+0000","last_change":"2024-03-28T17:58:54.017029+0000","last_active":"2024-03-28T17:58:55.096368+0000","last_peered":"2024-03-28T17:58:55.096368+0000","last_clean":"2024-03-28T17:58:55.096368+0000","last_became_active":"2024-03-28T17:58:54.014826+0000","last_became_peered":"2024-03-28T17:58:54.014826+0000","last_unstale":"2024-03-28T17:58:55.096368+0000","last_undegraded":"2024-03-28T17:58:55.096368+0000","last_fullsized":"2024-03-28T17:58:55.096368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019932+0000","last_change":"2024-03-28T17:58:53.005570+0000","last_active":"2024-03-28T17:58:55.019932+0000","last_peered":"2024-03-28T17:58:55.019932+0000","last_clean":"2024-03-28T17:58:55.019932+0000","last_became_active":"2024-03-28T17:58:53.005285+0000","last_became_peered":"2024-03-28T17:58:53.005285+0000","last_unstale":"2024-03-28T17:58:55.019932+0000","last_undegraded":"2024-03-28T17:58:55.019932+0000","last_fullsized":"2024-03-28T17:58:55.019932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098116+0000","last_change":"2024-03-28T17:58:54.018755+0000","last_active":"2024-03-28T17:58:55.098116+0000","last_peered":"2024-03-28T17:58:55.098116+0000","last_clean":"2024-03-28T17:58:55.098116+0000","last_became_active":"2024-03-28T17:58:54.018403+0000","last_became_peered":"2024-03-28T17:58:54.018403+0000","last_unstale":"2024-03-28T17:58:55.098116+0000","last_undegraded":"2024-03-28T17:58:55.098116+0000","last_fullsized":"2024-03-28T17:58:55.098116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095055+0000","last_change":"2024-03-28T17:58:53.008509+0000","last_active":"2024-03-28T17:58:55.095055+0000","last_peered":"2024-03-28T17:58:55.095055+0000","last_clean":"2024-03-28T17:58:55.095055+0000","last_became_active":"2024-03-28T17:58:53.008065+0000","last_became_peered":"2024-03-28T17:58:53.008065+0000","last_unstale":"2024-03-28T17:58:55.095055+0000","last_undegraded":"2024-03-28T17:58:55.095055+0000","last_fullsized":"2024-03-28T17:58:55.095055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095224+0000","last_change":"2024-03-28T17:58:54.018790+0000","last_active":"2024-03-28T17:58:55.095224+0000","last_peered":"2024-03-28T17:58:55.095224+0000","last_clean":"2024-03-28T17:58:55.095224+0000","last_became_active":"2024-03-28T17:58:54.018306+0000","last_became_peered":"2024-03-28T17:58:54.018306+0000","last_unstale":"2024-03-28T17:58:55.095224+0000","last_undegraded":"2024-03-28T17:58:55.095224+0000","last_fullsized":"2024-03-28T17:58:55.095224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:02.245546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996892+0000","last_change":"2024-03-28T17:58:53.007176+0000","last_active":"2024-03-28T17:58:54.996892+0000","last_peered":"2024-03-28T17:58:54.996892+0000","last_clean":"2024-03-28T17:58:54.996892+0000","last_became_active":"2024-03-28T17:58:53.002876+0000","last_became_peered":"2024-03-28T17:58:53.002876+0000","last_unstale":"2024-03-28T17:58:54.996892+0000","last_undegraded":"2024-03-28T17:58:54.996892+0000","last_fullsized":"2024-03-28T17:58:54.996892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095113+0000","last_change":"2024-03-28T17:58:54.021263+0000","last_active":"2024-03-28T17:58:55.095113+0000","last_peered":"2024-03-28T17:58:55.095113+0000","last_clean":"2024-03-28T17:58:55.095113+0000","last_became_active":"2024-03-28T17:58:54.015718+0000","last_became_peered":"2024-03-28T17:58:54.015718+0000","last_unstale":"2024-03-28T17:58:55.095113+0000","last_undegraded":"2024-03-28T17:58:55.095113+0000","last_fullsized":"2024-03-28T17:58:55.095113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997526+0000","last_change":"2024-03-28T17:58:53.000138+0000","last_active":"2024-03-28T17:58:54.997526+0000","last_peered":"2024-03-28T17:58:54.997526+0000","last_clean":"2024-03-28T17:58:54.997526+0000","last_became_active":"2024-03-28T17:58:52.999805+0000","last_became_peered":"2024-03-28T17:58:52.999805+0000","last_unstale":"2024-03-28T17:58:54.997526+0000","last_undegraded":"2024-03-28T17:58:54.997526+0000","last_fullsized":"2024-03-28T17:58:54.997526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095084+0000","last_change":"2024-03-28T17:58:54.006123+0000","last_active":"2024-03-28T17:58:55.095084+0000","last_peered":"2024-03-28T17:58:55.095084+0000","last_clean":"2024-03-28T17:58:55.095084+0000","last_became_active":"2024-03-28T17:58:54.005714+0000","last_became_peered":"2024-03-28T17:58:54.005714+0000","last_unstale":"2024-03-28T17:58:55.095084+0000","last_undegraded":"2024-03-28T17:58:55.095084+0000","last_fullsized":"2024-03-28T17:58:55.095084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095450+0000","last_change":"2024-03-28T17:58:53.009871+0000","last_active":"2024-03-28T17:58:55.095450+0000","last_peered":"2024-03-28T17:58:55.095450+0000","last_clean":"2024-03-28T17:58:55.095450+0000","last_became_active":"2024-03-28T17:58:53.009550+0000","last_became_peered":"2024-03-28T17:58:53.009550+0000","last_unstale":"2024-03-28T17:58:55.095450+0000","last_undegraded":"2024-03-28T17:58:55.095450+0000","last_fullsized":"2024-03-28T17:58:55.095450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:03.353429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019393+0000","last_change":"2024-03-28T17:58:54.020986+0000","last_active":"2024-03-28T17:58:55.019393+0000","last_peered":"2024-03-28T17:58:55.019393+0000","last_clean":"2024-03-28T17:58:55.019393+0000","last_became_active":"2024-03-28T17:58:54.020717+0000","last_became_peered":"2024-03-28T17:58:54.020717+0000","last_unstale":"2024-03-28T17:58:55.019393+0000","last_undegraded":"2024-03-28T17:58:55.019393+0000","last_fullsized":"2024-03-28T17:58:55.019393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999845+0000","last_change":"2024-03-28T17:58:54.022957+0000","last_active":"2024-03-28T17:58:54.999845+0000","last_peered":"2024-03-28T17:58:54.999845+0000","last_clean":"2024-03-28T17:58:54.999845+0000","last_became_active":"2024-03-28T17:58:54.022234+0000","last_became_peered":"2024-03-28T17:58:54.022234+0000","last_unstale":"2024-03-28T17:58:54.999845+0000","last_undegraded":"2024-03-28T17:58:54.999845+0000","last_fullsized":"2024-03-28T17:58:54.999845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097078+0000","last_change":"2024-03-28T17:58:53.010194+0000","last_active":"2024-03-28T17:58:55.097078+0000","last_peered":"2024-03-28T17:58:55.097078+0000","last_clean":"2024-03-28T17:58:55.097078+0000","last_became_active":"2024-03-28T17:58:53.009750+0000","last_became_peered":"2024-03-28T17:58:53.009750+0000","last_unstale":"2024-03-28T17:58:55.097078+0000","last_undegraded":"2024-03-28T17:58:55.097078+0000","last_fullsized":"2024-03-28T17:58:55.097078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000545+0000","last_change":"2024-03-28T17:58:54.016961+0000","last_active":"2024-03-28T17:58:55.000545+0000","last_peered":"2024-03-28T17:58:55.000545+0000","last_clean":"2024-03-28T17:58:55.000545+0000","last_became_active":"2024-03-28T17:58:54.016695+0000","last_became_peered":"2024-03-28T17:58:54.016695+0000","last_unstale":"2024-03-28T17:58:55.000545+0000","last_undegraded":"2024-03-28T17:58:55.000545+0000","last_fullsized":"2024-03-28T17:58:55.000545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095647+0000","last_change":"2024-03-28T17:58:53.011900+0000","last_active":"2024-03-28T17:58:55.095647+0000","last_peered":"2024-03-28T17:58:55.095647+0000","last_clean":"2024-03-28T17:58:55.095647+0000","last_became_active":"2024-03-28T17:58:53.011463+0000","last_became_peered":"2024-03-28T17:58:53.011463+0000","last_unstale":"2024-03-28T17:58:55.095647+0000","last_undegraded":"2024-03-28T17:58:55.095647+0000","last_fullsized":"2024-03-28T17:58:55.095647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098095+0000","last_change":"2024-03-28T17:58:54.010789+0000","last_active":"2024-03-28T17:58:55.098095+0000","last_peered":"2024-03-28T17:58:55.098095+0000","last_clean":"2024-03-28T17:58:55.098095+0000","last_became_active":"2024-03-28T17:58:54.010520+0000","last_became_peered":"2024-03-28T17:58:54.010520+0000","last_unstale":"2024-03-28T17:58:55.098095+0000","last_undegraded":"2024-03-28T17:58:55.098095+0000","last_fullsized":"2024-03-28T17:58:55.098095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095653+0000","last_change":"2024-03-28T17:58:52.999999+0000","last_active":"2024-03-28T17:58:55.095653+0000","last_peered":"2024-03-28T17:58:55.095653+0000","last_clean":"2024-03-28T17:58:55.095653+0000","last_became_active":"2024-03-28T17:58:52.999659+0000","last_became_peered":"2024-03-28T17:58:52.999659+0000","last_unstale":"2024-03-28T17:58:55.095653+0000","last_undegraded":"2024-03-28T17:58:55.095653+0000","last_fullsized":"2024-03-28T17:58:55.095653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095619+0000","last_change":"2024-03-28T17:58:54.010224+0000","last_active":"2024-03-28T17:58:55.095619+0000","last_peered":"2024-03-28T17:58:55.095619+0000","last_clean":"2024-03-28T17:58:55.095619+0000","last_became_active":"2024-03-28T17:58:54.008396+0000","last_became_peered":"2024-03-28T17:58:54.008396+0000","last_unstale":"2024-03-28T17:58:55.095619+0000","last_undegraded":"2024-03-28T17:58:55.095619+0000","last_fullsized":"2024-03-28T17:58:55.095619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098248+0000","last_change":"2024-03-28T17:58:53.008301+0000","last_active":"2024-03-28T17:58:55.098248+0000","last_peered":"2024-03-28T17:58:55.098248+0000","last_clean":"2024-03-28T17:58:55.098248+0000","last_became_active":"2024-03-28T17:58:53.007920+0000","last_became_peered":"2024-03-28T17:58:53.007920+0000","last_unstale":"2024-03-28T17:58:55.098248+0000","last_undegraded":"2024-03-28T17:58:55.098248+0000","last_fullsized":"2024-03-28T17:58:55.098248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T17:58:58.976 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T17:58:58.976 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T17:58:58.976 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-28T17:58:58.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-28T17:58:58.977 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-28T17:58:59.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.230+0000 7f5c4ba00640 1 Processor -- start 2024-03-28T17:58:59.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.230+0000 7f5c4ba00640 1 -- start start 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c4ba00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c440cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c4ba00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c440ced30 0x7f5c4406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c4ba00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4406b240 con 0x7f5c440cf9d0 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c4ba00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4406bb70 con 0x7f5c440ced30 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c4ba00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4406b520 con 0x7f5c44161930 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c43e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5c440cf9d0 0x7f5c440ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56558/0 (socket says 172.21.15.62:56558) 2024-03-28T17:58:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.231+0000 7f5c43e00640 1 -- 172.21.15.62:0/3509490951 learned_addr learned my addr 172.21.15.62:0/3509490951 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:58:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c49000640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c440cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3509490951 >> v2:172.21.15.81:3300/0 conn(0x7f5c440ced30 0x7f5c4406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824424522 0 0) 0x7f5c4406b240 con 0x7f5c440cf9d0 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c28003520 con 0x7f5c440cf9d0 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242156057 0 0) 0x7f5c28003520 con 0x7f5c440cf9d0 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 msgr2=0x7f5c440cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c440cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 >> v2:172.21.15.81:3300/0 conn(0x7f5c440ced30 msgr2=0x7f5c4406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 --2- 172.21.15.62:0/3509490951 >> v2:172.21.15.81:3300/0 conn(0x7f5c440ced30 0x7f5c4406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4406b800 con 0x7f5c440cf9d0 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f5c34001670 con 0x7f5c440cf9d0 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 >> v1:172.21.15.62:6789/0 conn(0x7f5c440cf9d0 legacy=0x7f5c440ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c280039b0 0x7f5c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c28004800 0x7f5c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4406bb70 con 0x7f5c280039b0 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4406b520 con 0x7f5c28004800 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.232+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2800b9c0 con 0x7f5c44161930 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c280039b0 0x7f5c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c43e00640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c28004800 0x7f5c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 -- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 msgr2=0x7f5c280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 -- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c28004800 msgr2=0x7f5c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c28004800 0x7f5c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49000640 1 -- 172.21.15.62:0/3509490951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28003520 con 0x7f5c280039b0 2024-03-28T17:58:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.233+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c44161930 0x7f5c280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.234+0000 7f5c49000640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c280039b0 0x7f5c28003dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ac0 tx=0x7f5c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec91707bce9dbca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c039070 con 0x7f5c280039b0 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c02c6b0 con 0x7f5c280039b0 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c43400640 1 -- 172.21.15.62:0/3509490951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c009690 con 0x7f5c280039b0 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3509490951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c280039b0 msgr2=0x7f5c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3509490951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c280039b0 0x7f5c28003dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ac0 tx=0x7f5c28005830 comp rx=0 tx=0).stop 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c49a00640 1 -- 172.21.15.62:0/3509490951 reap_dead start 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3509490951 shutdown_connections 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3509490951 >> 172.21.15.62:0/3509490951 conn(0x7f5c440b38e0 msgr2=0x7f5c440b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3509490951 shutdown_connections 2024-03-28T17:58:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.235+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3509490951 wait complete. 2024-03-28T17:58:59.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.237+0000 7f5c4ba00640 1 Processor -- start 2024-03-28T17:58:59.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.237+0000 7f5c4ba00640 1 -- start start 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c4415ba60 0x7f5c440b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c4415c180 0x7f5c4415fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440cd840 con 0x7f5c44160070 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440cd6c0 con 0x7f5c4415c180 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c4ba00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440cd9c0 con 0x7f5c4415ba60 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c49a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c49a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46802/0 (socket says 172.21.15.62:46802) 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.238+0000 7f5c49a00640 1 -- 172.21.15.62:0/3610399797 learned_addr learned my addr 172.21.15.62:0/3610399797 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49000640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c4415ba60 0x7f5c440b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c43e00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c4415c180 0x7f5c4415fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 -- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c4415ba60 msgr2=0x7f5c440b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c4415ba60 0x7f5c440b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 -- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c4415c180 msgr2=0x7f5c4415fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c4415c180 0x7f5c4415fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 -- 172.21.15.62:0/3610399797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440b6480 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c49a00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c440b37a0 tx=0x7f5c3c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c01c070 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c017400 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0177f0 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.239+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4409f630 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.240+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4415be40 con 0x7f5c44160070 2024-03-28T17:58:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.241+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c441604f0 con 0x7f5c44160070 2024-03-28T17:58:59.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.241+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5c3c005d10 con 0x7f5c44160070 2024-03-28T17:58:59.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.242+0000 7f5c40c00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 0x7f5c2006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.242+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c3c08aba0 con 0x7f5c44160070 2024-03-28T17:58:59.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.243+0000 7f5c49000640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 0x7f5c2006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.244+0000 7f5c49000640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 0x7f5c2006e0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c036000 tx=0x7f5c2c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.247+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5c3c012070 con 0x7f5c44160070 2024-03-28T17:58:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.500+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5c4409fb50 con 0x7f5c44160070 2024-03-28T17:58:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.504+0000 7f5c40c00640 1 -- 172.21.15.62:0/3610399797 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+172578 (secure 0 0 0) 0x7f5c4409fb50 con 0x7f5c44160070 2024-03-28T17:58:59.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:59.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 msgr2=0x7f5c2006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 0x7f5c2006e0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c036000 tx=0x7f5c2c030000 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 msgr2=0x7f5c440b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c440b37a0 tx=0x7f5c3c00f4a0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 shutdown_connections 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c2006bc00 0x7f5c2006e0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c44160070 0x7f5c440b4820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c4415c180 0x7f5c4415fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 --2- 172.21.15.62:0/3610399797 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c4415ba60 0x7f5c440b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 >> 172.21.15.62:0/3610399797 conn(0x7f5c440b38e0 msgr2=0x7f5c4406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 shutdown_connections 2024-03-28T17:58:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.509+0000 7f5c4ba00640 1 -- 172.21.15.62:0/3610399797 wait complete. 2024-03-28T17:58:59.519 INFO:tasks.ceph.mds.d.smithi081.stderr:2024-03-28T17:58:59.518+0000 7fdba1e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T17:58:59.527 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1316232289},{"type":"v1","addr":"172.21.15.81:6833","nonce":1316232289}]},"active_addr":"172.21.15.81:6833/1316232289","active_change":"2024-03-28T17:58:34.286621+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3823360532}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":893948476}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3928607612}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3484324637}]}]} 2024-03-28T17:58:59.530 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-28T17:58:59.531 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-28T17:58:59.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:58:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.747+0000 7f8436800640 1 Processor -- start 2024-03-28T17:58:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.747+0000 7f8436800640 1 -- start start 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8436800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8436800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 0x7f843015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8436800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84300cebb0 con 0x7f843006cd80 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8436800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300cee90 con 0x7f8430072ab0 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8436800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cf170 con 0x7f84301456d0 2024-03-28T17:58:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8435400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8434a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 0x7f843015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8435400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54084/0 (socket says 172.21.15.62:54084) 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8434a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 0x7f843015fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51304/0 (socket says 172.21.15.62:51304) 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.749+0000 7f8435400640 1 -- 172.21.15.62:0/4080180176 learned_addr learned my addr 172.21.15.62:0/4080180176 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.750+0000 7f8435400640 1 -- 172.21.15.62:0/4080180176 >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 msgr2=0x7f843015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.750+0000 7f8435400640 1 --2- 172.21.15.62:0/4080180176 >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 0x7f843015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.750+0000 7f8435400640 1 -- 172.21.15.62:0/4080180176 >> v1:172.21.15.81:6789/0 conn(0x7f843006cd80 legacy=0x7f843006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.750+0000 7f8435400640 1 -- 172.21.15.62:0/4080180176 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300cf450 con 0x7f84301456d0 2024-03-28T17:58:59.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.751+0000 7f8435400640 1 --2- 172.21.15.62:0/4080180176 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8424009870 tx=0x7f842402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fea44a9db0faab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f842fe00640 1 -- 172.21.15.62:0/4080180176 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424035020 con 0x7f84301456d0 2024-03-28T17:58:59.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f842fe00640 1 -- 172.21.15.62:0/4080180176 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424031640 con 0x7f84301456d0 2024-03-28T17:58:59.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f842fe00640 1 -- 172.21.15.62:0/4080180176 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842402d040 con 0x7f84301456d0 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 -- 172.21.15.62:0/4080180176 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 msgr2=0x7f843015f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 --2- 172.21.15.62:0/4080180176 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8424009870 tx=0x7f842402b8f0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 -- 172.21.15.62:0/4080180176 shutdown_connections 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 --2- 172.21.15.62:0/4080180176 >> v2:172.21.15.81:3300/0 conn(0x7f8430072ab0 0x7f843015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 --2- 172.21.15.62:0/4080180176 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f84301456d0 0x7f843015f980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.752+0000 7f8436800640 1 -- 172.21.15.62:0/4080180176 >> 172.21.15.62:0/4080180176 conn(0x7f84300b1730 msgr2=0x7f84300b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.753+0000 7f8436800640 1 -- 172.21.15.62:0/4080180176 shutdown_connections 2024-03-28T17:58:59.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.753+0000 7f8436800640 1 -- 172.21.15.62:0/4080180176 wait complete. 2024-03-28T17:58:59.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.754+0000 7f8436800640 1 Processor -- start 2024-03-28T17:58:59.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.754+0000 7f8436800640 1 -- start start 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f843006cd80 0x7f843013ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8430072ab0 0x7f84301e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430071a50 con 0x7f84301456d0 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cf730 con 0x7f843006cd80 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.755+0000 7f8436800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8430071bd0 con 0x7f8430072ab0 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46816/0 (socket says 172.21.15.62:46816) 2024-03-28T17:58:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 -- 172.21.15.62:0/2823543651 learned_addr learned my addr 172.21.15.62:0/2823543651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435400640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f843006cd80 0x7f843013ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8434a00640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8430072ab0 0x7f84301e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 -- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8430072ab0 msgr2=0x7f84301e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8430072ab0 0x7f84301e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 -- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f843006cd80 msgr2=0x7f843013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f843006cd80 0x7f843013ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 -- 172.21.15.62:0/2823543651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84301e5700 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.756+0000 7f8435e00640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f842800c7d0 tx=0x7f842800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.757+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428003b70 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.757+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428003d30 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.757+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842800aa90 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.757+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84300ae4f0 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.757+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84300aeab0 con 0x7f84301456d0 2024-03-28T17:58:59.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.758+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8430073490 con 0x7f84301456d0 2024-03-28T17:58:59.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.759+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f842800ac30 con 0x7f84301456d0 2024-03-28T17:58:59.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.760+0000 7f842d600640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 0x7f841006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:59.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.760+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f842808a9b0 con 0x7f84301456d0 2024-03-28T17:58:59.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.760+0000 7f8435400640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 0x7f841006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:59.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.761+0000 7f8435400640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 0x7f841006e090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8424036000 tx=0x7f842403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:59.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.765+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8428059490 con 0x7f84301456d0 2024-03-28T17:58:59.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.971+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f843004f4e0 con 0x7f84301456d0 2024-03-28T17:58:59.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.974+0000 7f842d600640 1 -- 172.21.15.62:0/2823543651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f8428058d90 con 0x7f84301456d0 2024-03-28T17:58:59.974 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:58:59.975 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.976+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 msgr2=0x7f841006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.976+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 0x7f841006e090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8424036000 tx=0x7f842403c040 comp rx=0 tx=0).stop 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.976+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 msgr2=0x7f843015da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.976+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f842800c7d0 tx=0x7f842800cc90 comp rx=0 tx=0).stop 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 shutdown_connections 2024-03-28T17:58:59.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f841006bbd0 0x7f841006e090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84301456d0 0x7f843015da60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8430072ab0 0x7f84301e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 --2- 172.21.15.62:0/2823543651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f843006cd80 0x7f843013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 >> 172.21.15.62:0/2823543651 conn(0x7f84300b1730 msgr2=0x7f84300cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 shutdown_connections 2024-03-28T17:58:59.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:58:59.977+0000 7f8436800640 1 -- 172.21.15.62:0/2823543651 wait complete. 2024-03-28T17:58:59.997 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-28T17:58:59.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.210+0000 7f74aca00640 1 Processor -- start 2024-03-28T17:59:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.210+0000 7f74aca00640 1 -- start start 2024-03-28T17:59:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.212+0000 7f74aca00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 0x7f74a8072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.212+0000 7f74aca00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f74a806caa0 0x7f74a806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.212+0000 7f74aca00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a8071ed0 con 0x7f74a815b9c0 2024-03-28T17:59:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.212+0000 7f74aca00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a8072c50 con 0x7f74a806caa0 2024-03-28T17:59:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.212+0000 7f74aca00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8071330 con 0x7f74a80722b0 2024-03-28T17:59:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.213+0000 7f74a7400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74a815b9c0 0x7f74a80717c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33700/0 (socket says 172.21.15.62:33700) 2024-03-28T17:59:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.213+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 0x7f74a8072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.213+0000 7f74a7400640 1 -- 172.21.15.62:0/2098725199 learned_addr learned my addr 172.21.15.62:0/2098725199 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.213+0000 7f74a7e00640 1 --2- 172.21.15.62:0/2098725199 >> v2:172.21.15.81:3300/0 conn(0x7f74a806caa0 0x7f74a806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a6000640 1 -- 172.21.15.62:0/2098725199 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272343830 0 0) 0x7f74a8071ed0 con 0x7f74a815b9c0 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a6000640 1 -- 172.21.15.62:0/2098725199 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f748c003520 con 0x7f74a815b9c0 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a7e00640 1 -- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 msgr2=0x7f74a8072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a7e00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 0x7f74a8072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a7e00640 1 -- 172.21.15.62:0/2098725199 >> v1:172.21.15.81:6789/0 conn(0x7f74a815b9c0 legacy=0x7f74a80717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a7e00640 1 -- 172.21.15.62:0/2098725199 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8070a40 con 0x7f74a806caa0 2024-03-28T17:59:00.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.214+0000 7f74a6a00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 0x7f74a8072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.215+0000 7f74a7e00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a806caa0 0x7f74a806cf30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a133dac6242560 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74a6000640 1 -- 172.21.15.62:0/2098725199 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0039070 con 0x7f74a806caa0 2024-03-28T17:59:00.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74a6000640 1 -- 172.21.15.62:0/2098725199 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a002cc50 con 0x7f74a806caa0 2024-03-28T17:59:00.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74a6000640 1 -- 172.21.15.62:0/2098725199 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0034440 con 0x7f74a806caa0 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 -- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a806caa0 msgr2=0x7f74a806cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a806caa0 0x7f74a806cf30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a002bea0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 -- 172.21.15.62:0/2098725199 shutdown_connections 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a806caa0 0x7f74a806cf30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 --2- 172.21.15.62:0/2098725199 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a80722b0 0x7f74a8072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.216+0000 7f74aca00640 1 -- 172.21.15.62:0/2098725199 >> 172.21.15.62:0/2098725199 conn(0x7f74a80b1730 msgr2=0x7f74a80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.217+0000 7f74aca00640 1 -- 172.21.15.62:0/2098725199 shutdown_connections 2024-03-28T17:59:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.217+0000 7f74aca00640 1 -- 172.21.15.62:0/2098725199 wait complete. 2024-03-28T17:59:00.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.218+0000 7f74aca00640 1 Processor -- start 2024-03-28T17:59:00.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.218+0000 7f74aca00640 1 -- start start 2024-03-28T17:59:00.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a806caa0 0x7f74a814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a80722b0 0x7f74a815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80b3810 con 0x7f74a815b9c0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80b3af0 con 0x7f74a80722b0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.219+0000 7f74aca00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8070ce0 con 0x7f74a806caa0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46824/0 (socket says 172.21.15.62:46824) 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 -- 172.21.15.62:0/1069651909 learned_addr learned my addr 172.21.15.62:0/1069651909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7400640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a806caa0 0x7f74a814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a6a00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a80722b0 0x7f74a815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 -- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a806caa0 msgr2=0x7f74a814c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a806caa0 0x7f74a814c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 -- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a80722b0 msgr2=0x7f74a815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a80722b0 0x7f74a815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.220+0000 7f74a7e00640 1 -- 172.21.15.62:0/1069651909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a815dcf0 con 0x7f74a815b9c0 2024-03-28T17:59:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f74a7e00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0039050 con 0x7f74a815b9c0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a0045070 con 0x7f74a815b9c0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0009450 con 0x7f74a815b9c0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a80cc410 con 0x7f74a815b9c0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.221+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80cc9d0 con 0x7f74a815b9c0 2024-03-28T17:59:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.222+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a806b370 con 0x7f74a815b9c0 2024-03-28T17:59:00.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.223+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f74a003e610 con 0x7f74a815b9c0 2024-03-28T17:59:00.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.224+0000 7f749f400640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 0x7f748806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.225+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74a00ad4c0 con 0x7f74a815b9c0 2024-03-28T17:59:00.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.225+0000 7f74a7400640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 0x7f748806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.226+0000 7f74a7400640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 0x7f748806e090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7498005880 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.229+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f74a007c060 con 0x7f74a815b9c0 2024-03-28T17:59:00.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.433+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74a8073490 con 0x7f74a815b9c0 2024-03-28T17:59:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.435+0000 7f749f400640 1 -- 172.21.15.62:0/1069651909 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f74a007b960 con 0x7f74a815b9c0 2024-03-28T17:59:00.435 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:00.435 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 msgr2=0x7f748806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 0x7f748806e090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7498005880 tx=0x7f7498009040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 msgr2=0x7f74a815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009d90 tx=0x7f74a0004a40 comp rx=0 tx=0).stop 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 shutdown_connections 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f748806bbd0 0x7f748806e090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a815b9c0 0x7f74a815d670 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74a80722b0 0x7f74a815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 --2- 172.21.15.62:0/1069651909 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f74a806caa0 0x7f74a814c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 >> 172.21.15.62:0/1069651909 conn(0x7f74a80b1730 msgr2=0x7f74a80d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.438+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 shutdown_connections 2024-03-28T17:59:00.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.439+0000 7f74aca00640 1 -- 172.21.15.62:0/1069651909 wait complete. 2024-03-28T17:59:00.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-28T17:59:00.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-28T17:59:00.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-28T17:59:00.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-28T17:59:00.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-28T17:59:00.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-28T17:59:00.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-28T17:59:00.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-28T17:59:00.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.770+0000 7fefd1c00640 1 Processor -- start 2024-03-28T17:59:00.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.771+0000 7fefd1c00640 1 -- start start 2024-03-28T17:59:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.773+0000 7fefd1c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fefc4005ed0 0x7fefc4006310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.773+0000 7fefd1c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4006850 0x7fefc4006ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.773+0000 7fefd1c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc4053710 con 0x7fefc40eacf0 2024-03-28T17:59:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.773+0000 7fefd1c00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc4054320 con 0x7fefc4005ed0 2024-03-28T17:59:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.773+0000 7fefd1c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40539f0 con 0x7fefc4006850 2024-03-28T17:59:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7f7d36600640 1 Processor -- start 2024-03-28T17:59:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7fefcaa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fefc40eacf0 0x7fefc404fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56560/0 (socket says 172.21.15.62:56560) 2024-03-28T17:59:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7fefcaa00640 1 -- 172.21.15.62:0/1466677147 learned_addr learned my addr 172.21.15.62:0/1466677147 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7fefca000640 1 --2- 172.21.15.62:0/1466677147 >> v2:172.21.15.81:3300/0 conn(0x7fefc4005ed0 0x7fefc4006310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7fefcb400640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4006850 0x7fefc4006ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.774+0000 7f7d36600640 1 -- start start 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.775+0000 7fefca000640 1 -- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4006850 msgr2=0x7fefc4006ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.775+0000 7fefca000640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4006850 0x7fefc4006ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.775+0000 7fefca000640 1 -- 172.21.15.62:0/1466677147 >> v1:172.21.15.62:6789/0 conn(0x7fefc40eacf0 legacy=0x7fefc404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.775+0000 7fd03a000640 1 Processor -- start 2024-03-28T17:59:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7fefca000640 1 -- 172.21.15.62:0/1466677147 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4053cd0 con 0x7fefc4005ed0 2024-03-28T17:59:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.775+0000 7fd03a000640 1 -- start start 2024-03-28T17:59:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.777+0000 7fd03a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340cee20 0x7fd0340727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd03a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0340aee30 0x7fd0340b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7f7d36600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d3015a110 0x7f7d300aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7f7d36600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7d300b4350 0x7f7d30070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7f7d36600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3006e8f0 con 0x7f7d300b4790 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7f7d36600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300b32b0 con 0x7f7d3015a110 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.776+0000 7f7d36600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300b3590 con 0x7f7d300b4350 2024-03-28T17:59:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.777+0000 7f7d2f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d3015a110 0x7f7d300aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.777+0000 7f7d2fe00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7d300b4790 0x7f7d3015a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33714/0 (socket says 172.21.15.62:33714) 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.777+0000 7f7d2fe00640 1 -- 172.21.15.62:0/2915760318 learned_addr learned my addr 172.21.15.62:0/2915760318 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7f5f32200640 1 Processor -- start 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd03a000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd034162b50 con 0x7fd0340b2950 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd03a000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd034163480 con 0x7fd0340aee30 2024-03-28T17:59:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd03a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd034162e30 con 0x7fd0340cee20 2024-03-28T17:59:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fb719000640 1 Processor -- start 2024-03-28T17:59:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fefca000640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4005ed0 0x7fefc4006310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87ede6a407f94b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7f7d2ea00640 1 --2- 172.21.15.62:0/2915760318 >> v2:172.21.15.81:3300/0 conn(0x7f7d300b4350 0x7f7d30070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fb719000640 1 -- start start 2024-03-28T17:59:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd033e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0340aee30 0x7fd0340b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd033e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0340aee30 0x7fd0340b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46836/0 (socket says 172.21.15.62:46836) 2024-03-28T17:59:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7fd033e00640 1 -- 172.21.15.62:0/3875130508 learned_addr learned my addr 172.21.15.62:0/3875130508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fd033400640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340cee20 0x7fd0340727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fefc9600640 1 -- 172.21.15.62:0/1466677147 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc035020 con 0x7fefc4005ed0 2024-03-28T17:59:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fefc9600640 1 -- 172.21.15.62:0/1466677147 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc031830 con 0x7fefc4005ed0 2024-03-28T17:59:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fd033e00640 1 -- 172.21.15.62:0/3875130508 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340cee20 msgr2=0x7fd0340727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fd033e00640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340cee20 0x7fd0340727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fd033e00640 1 -- 172.21.15.62:0/3875130508 >> v1:172.21.15.62:6789/0 conn(0x7fd0340b2950 legacy=0x7fd0340ae720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7f7d2f400640 1 -- 172.21.15.62:0/2915760318 >> v2:172.21.15.81:3300/0 conn(0x7f7d300b4350 msgr2=0x7f7d30070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7f7d2f400640 1 --2- 172.21.15.62:0/2915760318 >> v2:172.21.15.81:3300/0 conn(0x7f7d300b4350 0x7f7d30070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7f7d2f400640 1 -- 172.21.15.62:0/2915760318 >> v1:172.21.15.81:6789/0 conn(0x7f7d300b4790 legacy=0x7f7d3015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7f7d2f400640 1 -- 172.21.15.62:0/2915760318 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300b3870 con 0x7f7d3015a110 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.779+0000 7fefc9600640 1 -- 172.21.15.62:0/1466677147 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc031c40 con 0x7fefc4005ed0 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fb719000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7140b2f40 0x7fb7140d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fb719000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb71415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fb719000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140d3790 con 0x7fb7140d1960 2024-03-28T17:59:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fb719000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7140b71b0 con 0x7fb7140b2f40 2024-03-28T17:59:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fb719000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140b6ef0 con 0x7fb714071dd0 2024-03-28T17:59:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.778+0000 7f5f32200640 1 -- start start 2024-03-28T17:59:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fb712800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb71415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 -- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4005ed0 msgr2=0x7fefc4006310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.780+0000 7fd033e00640 1 -- 172.21.15.62:0/3875130508 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034163110 con 0x7fd0340aee30 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4005ed0 0x7fefc4006310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc02b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 -- 172.21.15.62:0/1466677147 shutdown_connections 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4006850 0x7fefc4006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1466677147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4005ed0 0x7fefc4006310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fefd1c00640 1 -- 172.21.15.62:0/1466677147 >> 172.21.15.62:0/1466677147 conn(0x7fefc4009fa0 msgr2=0x7fefc400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fb711400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb7140d1960 0x7fb7140716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56578/0 (socket says 172.21.15.62:56578) 2024-03-28T17:59:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fb711400640 1 -- 172.21.15.62:0/3672086179 learned_addr learned my addr 172.21.15.62:0/3672086179 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7fb711e00640 1 --2- 172.21.15.62:0/3672086179 >> v2:172.21.15.81:3300/0 conn(0x7fb7140b2f40 0x7fb7140d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7f5f32200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fd033e00640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340aee30 0x7fd0340b2f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009e50 tx=0x7fd02c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb30eb3804a88b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fefd1c00640 1 -- 172.21.15.62:0/1466677147 shutdown_connections 2024-03-28T17:59:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fefd1c00640 1 -- 172.21.15.62:0/1466677147 wait complete. 2024-03-28T17:59:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7f5f2b400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5f2c0b5630 0x7f5f2c0b5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33722/0 (socket says 172.21.15.62:33722) 2024-03-28T17:59:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7f5f2b400640 1 -- 172.21.15.62:0/2482028845 learned_addr learned my addr 172.21.15.62:0/2482028845 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7f5f2be00640 1 --2- 172.21.15.62:0/2482028845 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fd032000640 1 -- 172.21.15.62:0/3875130508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c02ca80 con 0x7fd0340aee30 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7fd032000640 1 -- 172.21.15.62:0/3875130508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c02cc20 con 0x7fd0340aee30 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fb711e00640 1 -- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 msgr2=0x7fb71415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fb711e00640 1 --2- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb71415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fb711e00640 1 -- 172.21.15.62:0/3672086179 >> v1:172.21.15.62:6789/0 conn(0x7fb7140d1960 legacy=0x7fb7140716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7fb711e00640 1 -- 172.21.15.62:0/3672086179 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140d2ef0 con 0x7fb7140b2f40 2024-03-28T17:59:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7fd032000640 1 -- 172.21.15.62:0/3875130508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0343f0 con 0x7fd0340aee30 2024-03-28T17:59:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.781+0000 7f7d2f400640 1 --2- 172.21.15.62:0/2915760318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3015a110 0x7f7d300aed00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d30067db0 tx=0x7f7d1c02fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b393dd82bb29e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.782+0000 7f7d2e000640 1 -- 172.21.15.62:0/2915760318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c03d070 con 0x7f7d3015a110 2024-03-28T17:59:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7f7d2e000640 1 -- 172.21.15.62:0/2915760318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c030ac0 con 0x7f7d3015a110 2024-03-28T17:59:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7f7d2e000640 1 -- 172.21.15.62:0/2915760318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c038440 con 0x7f7d3015a110 2024-03-28T17:59:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0d35f0 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c161990 con 0x7f5f2c0b5630 2024-03-28T17:59:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.783+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ce4b0 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 -- 172.21.15.62:0/2915760318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3015a110 msgr2=0x7f7d300aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 --2- 172.21.15.62:0/2915760318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3015a110 0x7f7d300aed00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d30067db0 tx=0x7f7d1c02fd10 comp rx=0 tx=0).stop 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f5f2a000640 1 -- 172.21.15.62:0/2482028845 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550522857 0 0) 0x7f5f2c0d35f0 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f5f2a000640 1 -- 172.21.15.62:0/2482028845 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f14003520 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 -- 172.21.15.62:0/2915760318 shutdown_connections 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 --2- 172.21.15.62:0/2915760318 >> v2:172.21.15.81:3300/0 conn(0x7f7d300b4350 0x7f7d30070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 --2- 172.21.15.62:0/2915760318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3015a110 0x7f7d300aed00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 -- 172.21.15.62:0/2915760318 >> 172.21.15.62:0/2915760318 conn(0x7f7d300b1650 msgr2=0x7f7d300b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7fefd1c00640 1 Processor -- start 2024-03-28T17:59:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f7d36600640 1 -- 172.21.15.62:0/2915760318 shutdown_connections 2024-03-28T17:59:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f5f2be00640 1 -- 172.21.15.62:0/2482028845 >> v1:172.21.15.81:6789/0 conn(0x7f5f2c0b5630 legacy=0x7f5f2c0b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f5f2be00640 1 -- 172.21.15.62:0/2482028845 >> v1:172.21.15.62:6789/0 conn(0x7f5f2c0d0d40 legacy=0x7f5f2c0d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.784+0000 7f5f2be00640 1 -- 172.21.15.62:0/2482028845 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0d3890 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.785+0000 7fefd1c00640 1 -- start start 2024-03-28T17:59:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.785+0000 7f7d36600640 1 -- 172.21.15.62:0/2915760318 wait complete. 2024-03-28T17:59:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.785+0000 7fd03a000640 1 -- 172.21.15.62:0/3875130508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340aee30 msgr2=0x7fd0340b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.785+0000 7fd03a000640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340aee30 0x7fd0340b2f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009e50 tx=0x7fd02c02bee0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fb711e00640 1 --2- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140b2f40 0x7fb7140d2850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb704009d90 tx=0x7fb70402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a093a4c3f97706 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fd03a000640 1 -- 172.21.15.62:0/3875130508 shutdown_connections 2024-03-28T17:59:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fd03a000640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340aee30 0x7fd0340b2f60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fd03a000640 1 --2- 172.21.15.62:0/3875130508 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340cee20 0x7fd0340727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fd03a000640 1 -- 172.21.15.62:0/3875130508 >> 172.21.15.62:0/3875130508 conn(0x7fd0340b1650 msgr2=0x7fd0340b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fe411c00640 1 Processor -- start 2024-03-28T17:59:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7f5f2be00640 1 --2- 172.21.15.62:0/2482028845 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c0b29d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097d0 tx=0x7f5f2002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0243033ef24537e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fe411c00640 1 -- start start 2024-03-28T17:59:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.786+0000 7fefd1c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4005ed0 0x7fefc40dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefd1c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4006850 0x7fefc40dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefd1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 0x7fefc4045570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefd1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40547b0 con 0x7fefc40eacf0 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefd1c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc4050190 con 0x7fefc4006850 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefd1c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc4054930 con 0x7fefc4005ed0 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fd03a000640 1 -- 172.21.15.62:0/3875130508 shutdown_connections 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fb710a00640 1 -- 172.21.15.62:0/3672086179 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704039070 con 0x7fb7140b2f40 2024-03-28T17:59:00.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fb710a00640 1 -- 172.21.15.62:0/3672086179 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70402ccf0 con 0x7fb7140b2f40 2024-03-28T17:59:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fb710a00640 1 -- 172.21.15.62:0/3672086179 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704034660 con 0x7fb7140b2f40 2024-03-28T17:59:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefcb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 0x7fefc4045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefcb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 0x7fefc4045570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46852/0 (socket says 172.21.15.62:46852) 2024-03-28T17:59:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7f7d36600640 1 Processor -- start 2024-03-28T17:59:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fd03a000640 1 -- 172.21.15.62:0/3875130508 wait complete. 2024-03-28T17:59:00.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.787+0000 7fefcb400640 1 -- 172.21.15.62:0/1511434355 learned_addr learned my addr 172.21.15.62:0/1511434355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcaa00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4005ed0 0x7fefc40dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 -- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140b2f40 msgr2=0x7fb7140d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 --2- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140b2f40 0x7fb7140d2850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb704009d90 tx=0x7fb70402bf40 comp rx=0 tx=0).stop 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 -- 172.21.15.62:0/3672086179 shutdown_connections 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 --2- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb71415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7f5f2a000640 1 -- 172.21.15.62:0/2482028845 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20035020 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7f5f2a000640 1 -- 172.21.15.62:0/2482028845 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20031810 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7f5f2a000640 1 -- 172.21.15.62:0/2482028845 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20031be0 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcb400640 1 -- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4005ed0 msgr2=0x7fefc40dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcb400640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefc4005ed0 0x7fefc40dbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcb400640 1 -- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4006850 msgr2=0x7fefc40dc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcb400640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefc4006850 0x7fefc40dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fefcb400640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc40dc500 con 0x7fefc40eacf0 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 --2- 172.21.15.62:0/3672086179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140b2f40 0x7fb7140d2850 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fb719000640 1 -- 172.21.15.62:0/3672086179 >> 172.21.15.62:0/3672086179 conn(0x7fb7140d3d90 msgr2=0x7fb7140d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fe411c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe40c0ce780 0x7fe40c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7fe411c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0d27b0 con 0x7fe40c15e210 2024-03-28T17:59:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7fe411c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0d2a90 con 0x7fe40c0d0b80 2024-03-28T17:59:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7fe411c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c0d2d70 con 0x7fe40c0ce780 2024-03-28T17:59:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7fefcb400640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 0x7fefc4045570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefcc069df0 tx=0x7fefcc073610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.788+0000 7f7d36600640 1 -- start start 2024-03-28T17:59:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7f2cfc600640 1 Processor -- start 2024-03-28T17:59:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7fb719000640 1 -- 172.21.15.62:0/3672086179 shutdown_connections 2024-03-28T17:59:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 msgr2=0x7f5f2c0b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7f2cfc600640 1 -- start start 2024-03-28T17:59:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7fe40a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe40c0ce780 0x7fe40c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 0x7f7d300cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d300b4790 0x7f7d300cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fe40b400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe40c0d0b80 0x7fe40c0b70e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33724/0 (socket says 172.21.15.62:33724) 2024-03-28T17:59:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fb719000640 1 -- 172.21.15.62:0/3672086179 wait complete. 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d3015a110 0x7f7d3014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b2f60 con 0x7f7d300b4350 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b3e30 con 0x7f7d300b4790 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d36600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300af240 con 0x7f7d3015a110 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.789+0000 7f5f32200640 1 --2- 172.21.15.62:0/2482028845 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c0b29d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097d0 tx=0x7f5f2002b8b0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 shutdown_connections 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f5f32200640 1 --2- 172.21.15.62:0/2482028845 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c0b29d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 >> 172.21.15.62:0/2482028845 conn(0x7f5f2c067400 msgr2=0x7f5f2c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc099070 con 0x7fefc40eacf0 2024-03-28T17:59:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc069580 con 0x7fefc40eacf0 2024-03-28T17:59:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc094630 con 0x7fefc40eacf0 2024-03-28T17:59:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fe40b400640 1 -- 172.21.15.62:0/257663466 learned_addr learned my addr 172.21.15.62:0/257663466 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fe40aa00640 1 --1- 172.21.15.62:0/257663466 >> v1:172.21.15.62:6789/0 conn(0x7fe40c15e210 0x7fe40c0d1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56592/0 (socket says 172.21.15.62:56592) 2024-03-28T17:59:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc404bb80 con 0x7fefc40eacf0 2024-03-28T17:59:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 0x7f7d300cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 0x7f7d300cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46872/0 (socket says 172.21.15.62:46872) 2024-03-28T17:59:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7fd03a000640 1 Processor -- start 2024-03-28T17:59:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc404c140 con 0x7fefc40eacf0 2024-03-28T17:59:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 shutdown_connections 2024-03-28T17:59:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d2f400640 1 -- 172.21.15.62:0/508191881 learned_addr learned my addr 172.21.15.62:0/508191881 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.790+0000 7f7d2fe00640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d3015a110 0x7f7d3014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7fd03a000640 1 -- start start 2024-03-28T17:59:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f5f32200640 1 -- 172.21.15.62:0/2482028845 wait complete. 2024-03-28T17:59:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7fe40a000640 1 -- 172.21.15.62:0/257663466 >> v1:172.21.15.81:6789/0 conn(0x7fe40c0d0b80 legacy=0x7fe40c0b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f2cfc600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2cf415e210 0x7f2cf40d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f2cfc600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2cf40d0d50 0x7f2cf40b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f2cfc600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40d2510 con 0x7f2cf40ce780 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f2cfc600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf40d27b0 con 0x7f2cf40d0d50 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f2cfc600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf40d2a90 con 0x7f2cf415e210 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f7d2f400640 1 -- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d3015a110 msgr2=0x7f7d3014e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f7d2f400640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7d3015a110 0x7f7d3014e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f7d2f400640 1 -- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d300b4790 msgr2=0x7f7d300cbb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f7d2f400640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d300b4790 0x7f7d300cbb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7f7d2f400640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3014c010 con 0x7f7d300b4350 2024-03-28T17:59:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.791+0000 7fe409600640 1 -- 172.21.15.62:0/257663466 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711483225 0 0) 0x7fe40c0d2a90 con 0x7fe40c0d0b80 2024-03-28T17:59:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fefc40052b0 con 0x7fefc40eacf0 2024-03-28T17:59:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7f289b400640 1 Processor -- start 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7f2cf9c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2cf415e210 0x7f2cf40d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7f2cf9c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2cf415e210 0x7f2cf40d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51346/0 (socket says 172.21.15.62:51346) 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7f2cf9c00640 1 -- 172.21.15.62:0/3281088653 learned_addr learned my addr 172.21.15.62:0/3281088653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 0x7fd03414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340b2950 0x7fd03415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340cee20 0x7fd03406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7f289b400640 1 -- start start 2024-03-28T17:59:00.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7f2cf9c00640 1 -- 172.21.15.62:0/3281088653 >> v2:172.21.15.62:3300/0 conn(0x7f2cf40d0d50 msgr2=0x7f2cf40b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7f2cf9c00640 1 --2- 172.21.15.62:0/3281088653 >> v2:172.21.15.62:3300/0 conn(0x7f2cf40d0d50 0x7f2cf40b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7f2cf9c00640 1 -- 172.21.15.62:0/3281088653 >> v1:172.21.15.81:6789/0 conn(0x7f2cf40ce780 legacy=0x7f2cf40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7f2cf9c00640 1 -- 172.21.15.62:0/3281088653 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40d2d70 con 0x7f2cf415e210 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340b34a0 con 0x7fd0340cee20 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341627d0 con 0x7fd0340aee30 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.793+0000 7fd03a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340b3620 con 0x7fd0340b2950 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd032a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340b2950 0x7fd03415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 0x7fd03414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7fe40a000640 1 -- 172.21.15.62:0/257663466 >> v1:172.21.15.62:6789/0 conn(0x7fe40c15e210 legacy=0x7fe40c0d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.792+0000 7fe40a000640 1 -- 172.21.15.62:0/257663466 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c0d3050 con 0x7fe40c0ce780 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd032a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340b2950 0x7fd03415c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54152/0 (socket says 172.21.15.62:54152) 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd032a00640 1 -- 172.21.15.62:0/1770036376 learned_addr learned my addr 172.21.15.62:0/1770036376 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 0x7fd03414cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51352/0 (socket says 172.21.15.62:51352) 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7f5f32200640 1 Processor -- start 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 -- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340b2950 msgr2=0x7fd03415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd0340b2950 0x7fd03415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 -- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340cee20 msgr2=0x7fd03406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fd033400640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0340cee20 0x7fd03406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fb719000640 1 Processor -- start 2024-03-28T17:59:00.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fb719000640 1 -- start start 2024-03-28T17:59:00.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7fe40a000640 1 --2- 172.21.15.62:0/257663466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0ce780 0x7fe40c0d0640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80097a0 tx=0x7fe3f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c344510467e7bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.794+0000 7f5f32200640 1 -- start start 2024-03-28T17:59:00.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f289b400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f289415e210 0x7f28940d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f289b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28940ce780 0x7f28940d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fd033400640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03415b480 con 0x7fd0340aee30 2024-03-28T17:59:00.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fe409600640 1 -- 172.21.15.62:0/257663466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8034070 con 0x7fe40c0ce780 2024-03-28T17:59:00.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f289b400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 0x7f28940b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f289b400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940d2510 con 0x7f28940ce780 2024-03-28T17:59:00.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f7d2f400640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 0x7f7d300cf090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fefcc0947d0 con 0x7fefc40eacf0 2024-03-28T17:59:00.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fe409600640 1 -- 172.21.15.62:0/257663466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8030610 con 0x7fe40c0ce780 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7fe409600640 1 -- 172.21.15.62:0/257663466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80309e0 con 0x7fe40c0ce780 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f2c0b5630 0x7f5f2c1faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 0x7f5f2c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b31f0 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b3070 con 0x7f5f2c0b5630 2024-03-28T17:59:00.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f32200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0d28a0 con 0x7f5f2c0cf0f0 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f2893e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28940ce780 0x7f28940d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f289b400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940d27b0 con 0x7f289415e210 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.795+0000 7f2898a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f289415e210 0x7f28940d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f289b400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d2a90 con 0x7f28940d0b80 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb7141583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7140b2f40 0x7fb71415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140d1960 0x7fb71415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f2898a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f289415e210 0x7f28940d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51354/0 (socket says 172.21.15.62:51354) 2024-03-28T17:59:00.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f2898a00640 1 -- 172.21.15.62:0/898812834 learned_addr learned my addr 172.21.15.62:0/898812834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140d4bc0 con 0x7fb7140b2f40 2024-03-28T17:59:00.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140d34b0 con 0x7fb7140d1960 2024-03-28T17:59:00.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb719000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140b3320 con 0x7fb714071dd0 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c03d030 con 0x7f7d300b4350 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0133e0 con 0x7f7d300b4350 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c038440 con 0x7f7d300b4350 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 0x7f5f2c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3014c2f0 con 0x7f7d300b4350 2024-03-28T17:59:00.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 0x7f5f2c1609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46928/0 (socket says 172.21.15.62:46928) 2024-03-28T17:59:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f5f2be00640 1 -- 172.21.15.62:0/3891095720 learned_addr learned my addr 172.21.15.62:0/3891095720 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb711400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7140b2f40 0x7fb71415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb711400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7140b2f40 0x7fb71415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46936/0 (socket says 172.21.15.62:46936) 2024-03-28T17:59:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7fb711400640 1 -- 172.21.15.62:0/310112732 learned_addr learned my addr 172.21.15.62:0/310112732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3014c8b0 con 0x7f7d300b4350 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7fd033400640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 0x7fd03414cd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c5c0 tx=0x7fd01c00ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7fb711e00640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb7141583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2b400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f2c0b5630 0x7f5f2c1faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2be00640 1 -- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 msgr2=0x7f5f2c15cfb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2be00640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5f2c0cf0f0 0x7f5f2c15cfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2be00640 1 -- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f2c0b5630 msgr2=0x7f5f2c1faaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2be00640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f2c0b5630 0x7f5f2c1faaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2be00640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c1f9770 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f5f2b400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f2c0b5630 0x7f5f2c1faaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.796+0000 7f2899400640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 0x7f28940b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2898a00640 1 -- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 msgr2=0x7f28940b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2898a00640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 0x7f28940b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2898a00640 1 -- 172.21.15.62:0/898812834 >> v2:172.21.15.62:3300/0 conn(0x7f28940ce780 msgr2=0x7f28940d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2898a00640 1 --2- 172.21.15.62:0/898812834 >> v2:172.21.15.62:3300/0 conn(0x7f28940ce780 0x7f28940d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2898a00640 1 -- 172.21.15.62:0/898812834 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940d2d70 con 0x7f289415e210 2024-03-28T17:59:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7f2899400640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 0x7f28940b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb712800640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140d1960 0x7fb71415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 -- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140d1960 msgr2=0x7fb71415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7140d1960 0x7fb71415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7fe411c00640 1 -- 172.21.15.62:0/257663466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0ce780 msgr2=0x7fe40c0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.797+0000 7fe411c00640 1 --2- 172.21.15.62:0/257663466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0ce780 0x7fe40c0d0640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80097a0 tx=0x7fe3f802b960 comp rx=0 tx=0).stop 2024-03-28T17:59:00.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 -- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7140b2f40 msgr2=0x7fb71415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7140b2f40 0x7fb71415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71415bd20 con 0x7fb714071dd0 2024-03-28T17:59:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7f2cf9c00640 1 --2- 172.21.15.62:0/3281088653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40d1fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4067be0 tx=0x7f2ce802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3abd35be4d32c41e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d300680c0 con 0x7f7d300b4350 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fe411c00640 1 -- 172.21.15.62:0/257663466 shutdown_connections 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fe411c00640 1 --2- 172.21.15.62:0/257663466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0ce780 0x7fe40c0d0640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fe411c00640 1 -- 172.21.15.62:0/257663466 >> 172.21.15.62:0/257663466 conn(0x7fe40c066dd0 msgr2=0x7fe40c0671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fe411c00640 1 -- 172.21.15.62:0/257663466 shutdown_connections 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7f2cf3e00640 1 -- 172.21.15.62:0/3281088653 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8034070 con 0x7f2cf415e210 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2cf3e00640 1 -- 172.21.15.62:0/3281088653 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce8030860 con 0x7f2cf415e210 2024-03-28T17:59:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2cf3e00640 1 -- 172.21.15.62:0/3281088653 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8030c30 con 0x7f2cf415e210 2024-03-28T17:59:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7f2898a00640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289415e210 0x7f28940d1fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2894067be0 tx=0x7f2880034120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de399328b2f5f94a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7fefc2a00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefac06bca0 0x7fefac06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2893400640 1 -- 172.21.15.62:0/898812834 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880041070 con 0x7f289415e210 2024-03-28T17:59:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2893400640 1 -- 172.21.15.62:0/898812834 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288003c440 con 0x7f289415e210 2024-03-28T17:59:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2893400640 1 -- 172.21.15.62:0/898812834 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288003c810 con 0x7f289415e210 2024-03-28T17:59:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefcc1055d0 con 0x7fefc40eacf0 2024-03-28T17:59:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2cfc600640 1 -- 172.21.15.62:0/3281088653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 msgr2=0x7f2cf40d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f2cfc600640 1 --2- 172.21.15.62:0/3281088653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40d1fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4067be0 tx=0x7f2ce802b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7f5f2be00640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 0x7f5f2c1609e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097d0 tx=0x7f5f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f2cfc600640 1 -- 172.21.15.62:0/3281088653 shutdown_connections 2024-03-28T17:59:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f2cfc600640 1 --2- 172.21.15.62:0/3281088653 >> v2:172.21.15.62:3300/0 conn(0x7f2cf40d0d50 0x7f2cf40b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f2cfc600640 1 --2- 172.21.15.62:0/3281088653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40d1fd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f2cfc600640 1 -- 172.21.15.62:0/3281088653 >> 172.21.15.62:0/3281088653 conn(0x7f2cf4066dd0 msgr2=0x7f2cf40671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fb711e00640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb7141583e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb704033000 tx=0x7fb704004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704044030 con 0x7fb714071dd0 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.799+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70402caf0 con 0x7fb714071dd0 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71415aed0 con 0x7fb714071dd0 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71415b480 con 0x7fb714071dd0 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20041070 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2003ac00 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704039040 con 0x7fb714071dd0 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20035040 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1f9a50 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1fa010 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f289b400640 1 -- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289415e210 msgr2=0x7f28940d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f289b400640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289415e210 0x7f28940d1fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2894067be0 tx=0x7f2880034120 comp rx=0 tx=0).stop 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7fefc2a00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] conn(0x7fefac073ec0 0x7fefac0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefcc08ee50 con 0x7fefac073ec0 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefcc10c050 con 0x7fefc40eacf0 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7fefcaa00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefac06bca0 0x7fefac06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c015070 con 0x7fd0340aee30 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7fd03a000640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03415b760 con 0x7fd0340aee30 2024-03-28T17:59:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7fd03a000640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03415bd20 con 0x7fd0340aee30 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.800+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7d1c048060 con 0x7f7d300b4350 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f7d27400640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7d1006bca0 0x7f7d1006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d1c0ac100 con 0x7f7d300b4350 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f7d27400640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] conn(0x7f7d10073f20 0x7f7d10076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d1c04be50 con 0x7f7d10073f20 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1c0ac4f0 con 0x7f7d300b4350 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f2cfc600640 1 -- 172.21.15.62:0/3281088653 shutdown_connections 2024-03-28T17:59:00.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7f2cfc600640 1 -- 172.21.15.62:0/3281088653 wait complete. 2024-03-28T17:59:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c004220 con 0x7fd0340aee30 2024-03-28T17:59:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c015070 con 0x7fd0340aee30 2024-03-28T17:59:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f7d2ea00640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7d1006bca0 0x7f7d1006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.798+0000 7fe411c00640 1 -- 172.21.15.62:0/257663466 wait complete. 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f7d2fe00640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] conn(0x7f7d10073f20 0x7f7d10076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.801+0000 7fe411c00640 1 Processor -- start 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f289b400640 1 -- 172.21.15.62:0/898812834 shutdown_connections 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f289b400640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f28940d0b80 0x7f28940b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f289b400640 1 --2- 172.21.15.62:0/898812834 >> v2:172.21.15.62:3300/0 conn(0x7f28940ce780 0x7f28940d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f289b400640 1 --2- 172.21.15.62:0/898812834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289415e210 0x7f28940d1fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f289b400640 1 -- 172.21.15.62:0/898812834 >> 172.21.15.62:0/898812834 conn(0x7f2894066dd0 msgr2=0x7f28940671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7fe411c00640 1 -- start start 2024-03-28T17:59:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.803+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7140680c0 con 0x7fb714071dd0 2024-03-28T17:59:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7fefca000640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] conn(0x7fefac073ec0 0x7fefac0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f2c04f400 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.802+0000 7fefca000640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] conn(0x7fefac073ec0 0x7fefac0762e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.803+0000 7fd03a000640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd00c001030 con 0x7fd0340aee30 2024-03-28T17:59:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.803+0000 7f2cfc600640 1 Processor -- start 2024-03-28T17:59:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7fefcaa00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefac06bca0 0x7fefac06e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefb8005880 tx=0x7fefb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7f7d2ea00640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7d1006bca0 0x7f7d1006e160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d3014dd30 tx=0x7f7d20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7f7d2fe00640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] conn(0x7f7d10073f20 0x7f7d10076340 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== osd.1 v2:172.21.15.62:6800/2984117190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fefcc08ee50 con 0x7fefac073ec0 2024-03-28T17:59:00.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7f289b400640 1 -- 172.21.15.62:0/898812834 shutdown_connections 2024-03-28T17:59:00.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5f2004c060 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7fe411c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe40c0ce780 0x7fe40c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7fe411c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 0x7fe40c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7fe411c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe40c15e210 0x7fe40c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== osd.3 v2:172.21.15.62:6824/1755400428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7d1c04be50 con 0x7f7d10073f20 2024-03-28T17:59:00.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7fe40a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 0x7fe40c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fe40a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 0x7fe40c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46942/0 (socket says 172.21.15.62:46942) 2024-03-28T17:59:00.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fe40a000640 1 -- 172.21.15.62:0/1479113229 learned_addr learned my addr 172.21.15.62:0/1479113229 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd01c003610 con 0x7fd0340aee30 2024-03-28T17:59:00.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.804+0000 7f289b400640 1 -- 172.21.15.62:0/898812834 wait complete. 2024-03-28T17:59:00.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb704047070 con 0x7fb714071dd0 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f5f1f400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5f0c06bbd0 0x7f5f0c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f200accb0 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f5f1f400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] conn(0x7f5f0c073df0 0x7f5f0c076210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f0c0768c0 con 0x7f5f0c073df0 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f200ad0a0 con 0x7f5f2c0d0d40 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.805+0000 7f2cfc600640 1 -- start start 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 0x7f2cf40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf40d0d50 0x7f2cf41554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf415e5f0 con 0x7f2cf40ce780 2024-03-28T17:59:00.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40d3330 con 0x7f2cf415e210 2024-03-28T17:59:00.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7f2cfc600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40b7620 con 0x7f2cf40d0d50 2024-03-28T17:59:00.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f289b400640 1 Processor -- start 2024-03-28T17:59:00.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f289b400640 1 -- start start 2024-03-28T17:59:00.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7f5f2b400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5f0c06bbd0 0x7f5f0c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15e5f0 con 0x7fe40c0d0b80 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0d1010 con 0x7fe40c0ce780 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.806+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b77f0 con 0x7fe40c15e210 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7fe40a000640 1 -- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe40c15e210 msgr2=0x7fe40c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7fe40a000640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe40c15e210 0x7fe40c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7fe40a000640 1 -- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe40c0ce780 msgr2=0x7fe40c15c300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7fe40a000640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe40c0ce780 0x7fe40c15c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.807+0000 7fe40a000640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c1f4770 con 0x7fe40c0d0b80 2024-03-28T17:59:00.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7fd02b400640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bdd0 0x7fd01006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7fb70a000640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb6f406bc00 0x7fb6f406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 0x7f28940cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28940d0b80 0x7f28940b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f289415e210 0x7f2894156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f5f2aa00640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] conn(0x7f5f0c073df0 0x7f5f0c076210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d1010 con 0x7f28940ce780 2024-03-28T17:59:00.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d3330 con 0x7f28940d0b80 2024-03-28T17:59:00.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f289b400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289415e5f0 con 0x7f289415e210 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7040acd50 con 0x7fb714071dd0 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f2cf9200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf40d0d50 0x7f2cf41554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f2cf9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 0x7f2cf40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f2cf9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 0x7f2cf40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46946/0 (socket says 172.21.15.62:46946) 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.808+0000 7f2cf9c00640 1 -- 172.21.15.62:0/621314125 learned_addr learned my addr 172.21.15.62:0/621314125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cfa600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 0x7f28940cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 0x7f28940cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46956/0 (socket says 172.21.15.62:46956) 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 -- 172.21.15.62:0/1633773251 learned_addr learned my addr 172.21.15.62:0/1633773251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2893e00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28940d0b80 0x7f28940b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7fb711400640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb6f406bc00 0x7fb6f406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7fb70a000640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] conn(0x7fb6f4073e00 0x7fb6f4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb70402de50 con 0x7fb6f4073e00 2024-03-28T17:59:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7040b2320 con 0x7fb714071dd0 2024-03-28T17:59:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2899400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f289415e210 0x7f2894156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd01c08a8e0 con 0x7fd0340aee30 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cf9c00640 1 -- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf40d0d50 msgr2=0x7f2cf41554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cf9c00640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2cf40d0d50 0x7f2cf41554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cf9c00640 1 -- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 msgr2=0x7f2cf40cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cf9c00640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cf415e210 0x7f2cf40cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2cf9c00640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40cc140 con 0x7f2cf40ce780 2024-03-28T17:59:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fe40a000640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 0x7fe40c0cd390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3f802bf70 tx=0x7fe3f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 -- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f289415e210 msgr2=0x7f2894156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f289415e210 0x7f2894156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 -- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28940d0b80 msgr2=0x7f28940b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.809+0000 7f2898a00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28940d0b80 0x7f28940b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7f2898a00640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28941609e0 con 0x7f28940ce780 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fd02b400640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] conn(0x7fd010073f70 0x7fd010076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7f2cf9c00640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 0x7f2cf40cbc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000c00 tx=0x7f2ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7f5f2b400640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5f0c06bbd0 0x7f5f0c06e090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005880 tx=0x7f5f18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd01c014e50 con 0x7fd010073f70 2024-03-28T17:59:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01c08ac90 con 0x7fd0340aee30 2024-03-28T17:59:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fd032a00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bdd0 0x7fd01006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fb712800640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] conn(0x7fb6f4073e00 0x7fb6f4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8034040 con 0x7fe40c0d0b80 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8039a50 con 0x7fe40c0d0b80 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8039e20 con 0x7fe40c0d0b80 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7f5f2aa00640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] conn(0x7f5f0c073df0 0x7f5f0c076210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.810+0000 7fd033e00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] conn(0x7fd010073f70 0x7fd010076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c1f49f0 con 0x7fe40c0d0b80 2024-03-28T17:59:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2898a00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 0x7f28940cd520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2880034160 tx=0x7f28800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8034040 con 0x7f2cf40ce780 2024-03-28T17:59:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce80433f0 con 0x7f2cf40ce780 2024-03-28T17:59:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80437e0 con 0x7f2cf40ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880041040 con 0x7f28940ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288004c070 con 0x7f28940ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288003a880 con 0x7f28940ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2894160140 con 0x7f28940ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2894160700 con 0x7f28940ce780 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.811+0000 7fb712800640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] conn(0x7fb6f4073e00 0x7fb6f4076220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.812+0000 7fb711400640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb6f406bc00 0x7fb6f406e0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b88f0 tx=0x7fb700009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.812+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c1f4fb0 con 0x7fe40c0d0b80 2024-03-28T17:59:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.813+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe40c1f5670 con 0x7fe40c0d0b80 2024-03-28T17:59:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.812+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== osd.5 v2:172.21.15.81:6800/619433227 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5f0c0768c0 con 0x7f5f0c073df0 2024-03-28T17:59:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.812+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf40cd3d0 con 0x7f2cf40ce780 2024-03-28T17:59:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.812+0000 7fd032a00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bdd0 0x7fd01006e290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd03415ff50 tx=0x7fd024007ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.813+0000 7fd033e00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] conn(0x7fd010073f70 0x7fd010076390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.814+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== osd.0 v2:172.21.15.62:6808/2494694084 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd01c014e50 con 0x7fd010073f70 2024-03-28T17:59:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.814+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28940680c0 con 0x7f28940ce780 2024-03-28T17:59:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.814+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== osd.2 v2:172.21.15.62:6816/2545705351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb70402de50 con 0x7fb6f4073e00 2024-03-28T17:59:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.815+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf40cb400 con 0x7f2cf40ce780 2024-03-28T17:59:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.815+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2ce8037070 con 0x7f2cf40ce780 2024-03-28T17:59:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.815+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f288003aa20 con 0x7f28940ce780 2024-03-28T17:59:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.816+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe3f8037070 con 0x7fe40c0d0b80 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.816+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cf40cb7e0 con 0x7f2cf40ce780 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2890c00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f287406bca0 0x7f287406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2cf1600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2cd406bc50 0x7f2cd406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ce80ac960 con 0x7f2cf40ce780 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2cf1600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] conn(0x7f2cd4073e70 0x7f2cd4076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ce802ce90 con 0x7f2cd4073e70 2024-03-28T17:59:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce80acd60 con 0x7f2cf40ce780 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28800b52a0 con 0x7f28940ce780 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2890c00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] conn(0x7f2874073ec0 0x7f28740762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2880036ea0 con 0x7f2874073ec0 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28800bc050 con 0x7f28940ce780 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.817+0000 7fe402a00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe3ec06bca0 0x7fe3ec06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3f80ad4b0 con 0x7fe40c0d0b80 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7fe402a00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] conn(0x7fe3ec073ec0 0x7fe3ec0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3f8001d30 con 0x7fe3ec073ec0 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f80ad8a0 con 0x7fe40c0d0b80 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7fe40aa00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe3ec06bca0 0x7fe3ec06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7f2cf9200640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2cd406bc50 0x7f2cd406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7f2cfa600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] conn(0x7f2cd4073e70 0x7f2cd4076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.818+0000 7f2893e00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f287406bca0 0x7f287406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.819+0000 7f2899400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] conn(0x7f2874073ec0 0x7f28740762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.819+0000 7fe40b400640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] conn(0x7fe3ec073ec0 0x7fe3ec0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.819+0000 7fe40aa00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe3ec06bca0 0x7fe3ec06e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0014d0 tx=0x7fe3fc0079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.820+0000 7f2cf9200640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2cd406bc50 0x7f2cd406e110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40b6e00 tx=0x7f2ce0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.820+0000 7f2899400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] conn(0x7f2874073ec0 0x7f28740762e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.820+0000 7f2893e00640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f287406bca0 0x7f287406e160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28940b9100 tx=0x7f2884009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.820+0000 7fe40b400640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] conn(0x7fe3ec073ec0 0x7fe3ec0762e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.821+0000 7f2cfa600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] conn(0x7f2cd4073e70 0x7f2cd4076290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.823+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== osd.4 v2:172.21.15.81:6824/2310676822 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe3f8001d30 con 0x7fe3ec073ec0 2024-03-28T17:59:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.823+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== osd.7 v2:172.21.15.81:6816/2473781039 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2880036ea0 con 0x7f2874073ec0 2024-03-28T17:59:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.825+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== osd.6 v2:172.21.15.81:6808/1688303827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2ce802ce90 con 0x7f2cd4073e70 2024-03-28T17:59:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.835+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 --> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d30156e70 con 0x7f7d10073f20 2024-03-28T17:59:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.839+0000 7f7d27400640 1 -- 172.21.15.62:0/508191881 <== osd.3 v2:172.21.15.62:6824/1755400428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d30156e70 con 0x7f7d10073f20 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] conn(0x7f7d10073f20 msgr2=0x7f7d10076340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:6824/1755400428,v1:172.21.15.62:6825/1755400428] conn(0x7f7d10073f20 0x7f7d10076340 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7d1006bca0 msgr2=0x7f7d1006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7d1006bca0 0x7f7d1006e160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d3014dd30 tx=0x7f7d20008040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 msgr2=0x7f7d300cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.840+0000 7f7d36600640 1 --2- 172.21.15.62:0/508191881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b4350 0x7f7d300cf090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f7d2fe00640 1 -- 172.21.15.62:0/508191881 reap_dead start 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 shutdown_connections 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 >> 172.21.15.62:0/508191881 conn(0x7f7d300b1650 msgr2=0x7f7d30157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 shutdown_connections 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f7d36600640 1 -- 172.21.15.62:0/508191881 wait complete. 2024-03-28T17:59:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.841+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 --> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f2c0ce9f0 con 0x7f5f0c073df0 2024-03-28T17:59:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.844+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 --> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fefc4038ef0 con 0x7fefac073ec0 2024-03-28T17:59:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.844+0000 7fefc2a00640 1 -- 172.21.15.62:0/1511434355 <== osd.1 v2:172.21.15.62:6800/2984117190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fefc4038ef0 con 0x7fefac073ec0 2024-03-28T17:59:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.844+0000 7f5f1f400640 1 -- 172.21.15.62:0/3891095720 <== osd.5 v2:172.21.15.81:6800/619433227 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5f2c0ce9f0 con 0x7f5f0c073df0 2024-03-28T17:59:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.846+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 --> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7140b8640 con 0x7fb6f4073e00 2024-03-28T17:59:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.846+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] conn(0x7fefac073ec0 msgr2=0x7fefac0762e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:6800/2984117190,v1:172.21.15.62:6801/2984117190] conn(0x7fefac073ec0 0x7fefac0762e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] conn(0x7f5f0c073df0 msgr2=0x7f5f0c076210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6800/619433227,v1:172.21.15.81:6801/619433227] conn(0x7f5f0c073df0 0x7f5f0c076210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5f0c06bbd0 msgr2=0x7f5f0c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5f0c06bbd0 0x7f5f0c06e090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005880 tx=0x7f5f18009040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 msgr2=0x7f5f2c1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 --2- 172.21.15.62:0/3891095720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f2c0d0d40 0x7f5f2c1609e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097d0 tx=0x7f5f200046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f2be00640 1 -- 172.21.15.62:0/3891095720 reap_dead start 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 --> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe40c06e2f0 con 0x7fe3ec073ec0 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefac06bca0 msgr2=0x7fefac06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefac06bca0 0x7fefac06e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefb8005880 tx=0x7fefb8009040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 msgr2=0x7fefc4045570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 --2- 172.21.15.62:0/1511434355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefc40eacf0 0x7fefc4045570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefcc069df0 tx=0x7fefcc073610 comp rx=0 tx=0).stop 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefcb400640 1 -- 172.21.15.62:0/1511434355 reap_dead start 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 shutdown_connections 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 >> 172.21.15.62:0/1511434355 conn(0x7fefc4009fa0 msgr2=0x7fefc404a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 shutdown_connections 2024-03-28T17:59:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.847+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 >> 172.21.15.62:0/3891095720 conn(0x7f5f2c067400 msgr2=0x7f5f2c0b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7fd03a000640 1 -- 172.21.15.62:0/1770036376 --> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd00c002dd0 con 0x7fd010073f70 2024-03-28T17:59:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 shutdown_connections 2024-03-28T17:59:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7f5f32200640 1 -- 172.21.15.62:0/3891095720 wait complete. 2024-03-28T17:59:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 shutdown_connections 2024-03-28T17:59:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7fefd1c00640 1 -- 172.21.15.62:0/1511434355 wait complete. 2024-03-28T17:59:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7fe402a00640 1 -- 172.21.15.62:0/1479113229 <== osd.4 v2:172.21.15.81:6824/2310676822 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe40c06e2f0 con 0x7fe3ec073ec0 2024-03-28T17:59:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.848+0000 7fd02b400640 1 -- 172.21.15.62:0/1770036376 <== osd.0 v2:172.21.15.62:6808/2494694084 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd00c002dd0 con 0x7fd010073f70 2024-03-28T17:59:00.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] conn(0x7fe3ec073ec0 msgr2=0x7fe3ec0762e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6824/2310676822,v1:172.21.15.81:6825/2310676822] conn(0x7fe3ec073ec0 0x7fe3ec0762e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe3ec06bca0 msgr2=0x7fe3ec06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fb70a000640 1 -- 172.21.15.62:0/310112732 <== osd.2 v2:172.21.15.62:6816/2545705351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb7140b8640 con 0x7fb6f4073e00 2024-03-28T17:59:00.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe3ec06bca0 0x7fe3ec06e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0014d0 tx=0x7fe3fc0079c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 msgr2=0x7fe40c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe411c00640 1 --2- 172.21.15.62:0/1479113229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe40c0d0b80 0x7fe40c0cd390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3f802bf70 tx=0x7fe3f80046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] conn(0x7fd010073f70 msgr2=0x7fd010076390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fd028c00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.62:6808/2494694084,v1:172.21.15.62:6809/2494694084] conn(0x7fd010073f70 0x7fd010076390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.849+0000 7fe40b400640 1 -- 172.21.15.62:0/1479113229 reap_dead start 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 shutdown_connections 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 >> 172.21.15.62:0/1479113229 conn(0x7fe40c066dd0 msgr2=0x7fe40c15f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 shutdown_connections 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bdd0 msgr2=0x7fd01006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bdd0 0x7fd01006e290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd03415ff50 tx=0x7fd024007ef0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 msgr2=0x7fd03414cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 --2- 172.21.15.62:0/1770036376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0340aee30 0x7fd03414cd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c5c0 tx=0x7fd01c00ca80 comp rx=0 tx=0).stop 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fe411c00640 1 -- 172.21.15.62:0/1479113229 wait complete. 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] conn(0x7fb6f4073e00 msgr2=0x7fb6f4076220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.62:6816/2545705351,v1:172.21.15.62:6817/2545705351] conn(0x7fb6f4073e00 0x7fb6f4076220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd033e00640 1 -- 172.21.15.62:0/1770036376 reap_dead start 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 shutdown_connections 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 >> 172.21.15.62:0/1770036376 conn(0x7fd0340b1650 msgr2=0x7fd034163910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb6f406bc00 msgr2=0x7fb6f406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb6f406bc00 0x7fb6f406e0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b88f0 tx=0x7fb700009040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 msgr2=0x7fb7141583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.850+0000 7fb719000640 1 --2- 172.21.15.62:0/310112732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb714071dd0 0x7fb7141583e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb704033000 tx=0x7fb704004a40 comp rx=0 tx=0).stop 2024-03-28T17:59:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.851+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 shutdown_connections 2024-03-28T17:59:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.851+0000 7fd028c00640 1 -- 172.21.15.62:0/1770036376 wait complete. 2024-03-28T17:59:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.851+0000 7fb712800640 1 -- 172.21.15.62:0/310112732 reap_dead start 2024-03-28T17:59:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.851+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 shutdown_connections 2024-03-28T17:59:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.851+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 >> 172.21.15.62:0/310112732 conn(0x7fb7140d3d90 msgr2=0x7fb7140b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.853+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 shutdown_connections 2024-03-28T17:59:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.853+0000 7fb719000640 1 -- 172.21.15.62:0/310112732 wait complete. 2024-03-28T17:59:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.855+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 --> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2cf40680c0 con 0x7f2cd4073e70 2024-03-28T17:59:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.856+0000 7f2cf1600640 1 -- 172.21.15.62:0/621314125 <== osd.6 v2:172.21.15.81:6808/1688303827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2cf40680c0 con 0x7f2cd4073e70 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] conn(0x7f2cd4073e70 msgr2=0x7f2cd4076290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6808/1688303827,v1:172.21.15.81:6809/1688303827] conn(0x7f2cd4073e70 0x7f2cd4076290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2cd406bc50 msgr2=0x7f2cd406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2cd406bc50 0x7f2cd406e110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40b6e00 tx=0x7f2ce0009040 comp rx=0 tx=0).stop 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 msgr2=0x7f2cf40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 --2- 172.21.15.62:0/621314125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf40ce780 0x7f2cf40cbc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000c00 tx=0x7f2ce80046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfa600640 1 -- 172.21.15.62:0/621314125 reap_dead start 2024-03-28T17:59:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 shutdown_connections 2024-03-28T17:59:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.857+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 >> 172.21.15.62:0/621314125 conn(0x7f2cf4066dd0 msgr2=0x7f2cf40b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.858+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 shutdown_connections 2024-03-28T17:59:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.858+0000 7f2cfc600640 1 -- 172.21.15.62:0/621314125 wait complete. 2024-03-28T17:59:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.860+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 --> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28940ce560 con 0x7f2874073ec0 2024-03-28T17:59:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.861+0000 7f2890c00640 1 -- 172.21.15.62:0/1633773251 <== osd.7 v2:172.21.15.81:6816/2473781039 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f28940ce560 con 0x7f2874073ec0 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] conn(0x7f2874073ec0 msgr2=0x7f28740762e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6816/2473781039,v1:172.21.15.81:6817/2473781039] conn(0x7f2874073ec0 0x7f28740762e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f287406bca0 msgr2=0x7f287406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f287406bca0 0x7f287406e160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28940b9100 tx=0x7f2884009190 comp rx=0 tx=0).stop 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 msgr2=0x7f28940cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 --2- 172.21.15.62:0/1633773251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28940ce780 0x7f28940cd520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2880034160 tx=0x7f28800046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f2899400640 1 -- 172.21.15.62:0/1633773251 reap_dead start 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.866+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 shutdown_connections 2024-03-28T17:59:00.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.867+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 >> 172.21.15.62:0/1633773251 conn(0x7f2894066dd0 msgr2=0x7f289415d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:00.884 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:00.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T17:59:00.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.867+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 shutdown_connections 2024-03-28T17:59:00.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:00.867+0000 7f289b400640 1 -- 172.21.15.62:0/1633773251 wait complete. 2024-03-28T17:59:00.886 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:00.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T17:59:00.887 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:00.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T17:59:00.888 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:00.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T17:59:00.889 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-03-28T17:59:00.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T17:59:00.890 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:00.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T17:59:00.891 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:00.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-28T17:59:00.892 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:00.892 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.216+0000 7f69cd200640 1 Processor -- start 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.216+0000 7f69cd200640 1 -- start start 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 0x7f69c80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c80b96e0 0x7f69c80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69c80b9170 0x7f69c8165f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c80b63c0 con 0x7f69c80b9170 2024-03-28T17:59:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c80b57b0 con 0x7f69c8165890 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.218+0000 7f69cd200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c80b5a90 con 0x7f69c80b96e0 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.219+0000 7f69c7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69c80b9170 0x7f69c8165f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.219+0000 7f69c7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69c80b9170 0x7f69c8165f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46960/0 (socket says 172.21.15.62:46960) 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.219+0000 7f69c7e00640 1 -- 172.21.15.62:0/1791735612 learned_addr learned my addr 172.21.15.62:0/1791735612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.219+0000 7f69c7400640 1 --2- 172.21.15.62:0/1791735612 >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 0x7f69c80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.219+0000 7f69c6a00640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c80b96e0 0x7f69c80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.220+0000 7f69c7e00640 1 -- 172.21.15.62:0/1791735612 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c80b96e0 msgr2=0x7f69c80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.220+0000 7f69c7e00640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c80b96e0 0x7f69c80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.220+0000 7f69c7e00640 1 -- 172.21.15.62:0/1791735612 >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 msgr2=0x7f69c80ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.220+0000 7f69c7e00640 1 --2- 172.21.15.62:0/1791735612 >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 0x7f69c80ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.220+0000 7f69c7e00640 1 -- 172.21.15.62:0/1791735612 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80b5d70 con 0x7f69c80b9170 2024-03-28T17:59:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.222+0000 7f69c7400640 1 --2- 172.21.15.62:0/1791735612 >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 0x7f69c80ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.223+0000 7f69c7e00640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8165f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f69c80d4d60 tx=0x7f69c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b796b61b347db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.224+0000 7f69c6000640 1 -- 172.21.15.62:0/1791735612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0034070 con 0x7f69c80b9170 2024-03-28T17:59:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.224+0000 7f69c6000640 1 -- 172.21.15.62:0/1791735612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0030610 con 0x7f69c80b9170 2024-03-28T17:59:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.224+0000 7f69c6000640 1 -- 172.21.15.62:0/1791735612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0030a20 con 0x7f69c80b9170 2024-03-28T17:59:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.226+0000 7f69cd200640 1 -- 172.21.15.62:0/1791735612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 msgr2=0x7f69c8165f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.226+0000 7f69cd200640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8165f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f69c80d4d60 tx=0x7f69c002b960 comp rx=0 tx=0).stop 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.226+0000 7f69cd200640 1 -- 172.21.15.62:0/1791735612 shutdown_connections 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8165f60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 --2- 172.21.15.62:0/1791735612 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c80b96e0 0x7f69c80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 --2- 172.21.15.62:0/1791735612 >> v2:172.21.15.81:3300/0 conn(0x7f69c8165890 0x7f69c80ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 -- 172.21.15.62:0/1791735612 >> 172.21.15.62:0/1791735612 conn(0x7f69c8071cf0 msgr2=0x7f69c8066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 -- 172.21.15.62:0/1791735612 shutdown_connections 2024-03-28T17:59:01.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.227+0000 7f69cd200640 1 -- 172.21.15.62:0/1791735612 wait complete. 2024-03-28T17:59:01.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.230+0000 7f69cd200640 1 Processor -- start 2024-03-28T17:59:01.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.230+0000 7f69cd200640 1 -- start start 2024-03-28T17:59:01.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.232+0000 7f69cd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.233+0000 7f69c7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.233+0000 7f69c7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46964/0 (socket says 172.21.15.62:46964) 2024-03-28T17:59:01.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.233+0000 7f69c7400640 1 -- 172.21.15.62:0/3974068696 learned_addr learned my addr 172.21.15.62:0/3974068696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.233+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 0x7f69c8146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.234+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c8165890 0x7f69c8160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f4be9600640 1 Processor -- start 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.234+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c80d3980 con 0x7f69c80b9170 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.234+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c8165c70 con 0x7f69c80b96e0 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.234+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c8145ca0 con 0x7f69c8165890 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.235+0000 7f69c7e00640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c8165890 0x7f69c8160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.235+0000 7f69c7400640 1 -- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c8165890 msgr2=0x7f69c8160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.235+0000 7f69c7400640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c8165890 0x7f69c8160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f69c7400640 1 -- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 msgr2=0x7f69c8146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.234+0000 7f69c6a00640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 0x7f69c8146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f69c7400640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 0x7f69c8146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f69c7400640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c8144df0 con 0x7f69c80b9170 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f69c6a00640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 0x7f69c8146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.236+0000 7f69c7400640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d7d0 tx=0x7f69b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.237+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0004030 con 0x7f69c80b9170 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.237+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c8145090 con 0x7f69c80b9170 2024-03-28T17:59:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.237+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c8145650 con 0x7f69c80b9170 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.238+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b00041d0 con 0x7f69c80b9170 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.238+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b000b6c0 con 0x7f69c80b9170 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.237+0000 7f4be9600640 1 -- start start 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.239+0000 7f4be9600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4be40b98b0 0x7f4be4161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.239+0000 7f4be9600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40ce650 con 0x7f4be40d1020 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.239+0000 7f4be9600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be4162ae0 con 0x7f4be415e180 2024-03-28T17:59:01.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.239+0000 7f4be9600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be415e560 con 0x7f4be40b98b0 2024-03-28T17:59:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.239+0000 7f4be2000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4be40d1020 0x7f4be40d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56594/0 (socket says 172.21.15.62:56594) 2024-03-28T17:59:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.240+0000 7f4be2000640 1 -- 172.21.15.62:0/690177403 learned_addr learned my addr 172.21.15.62:0/690177403 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.241+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a0005290 con 0x7f69c80b9170 2024-03-28T17:59:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.241+0000 7f4be3400640 1 --2- 172.21.15.62:0/690177403 >> v2:172.21.15.81:3300/0 conn(0x7f4be40b98b0 0x7f4be4161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.241+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194494528 0 0) 0x7f4be40ce650 con 0x7f4be40d1020 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.241+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8003520 con 0x7f4be40d1020 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.241+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640533830 0 0) 0x7f4be4162ae0 con 0x7f4be415e180 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be40ce650 con 0x7f4be415e180 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442973004 0 0) 0x7f4bc8003520 con 0x7f4be40d1020 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 >> v2:172.21.15.81:3300/0 conn(0x7f4be40b98b0 msgr2=0x7f4be4161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 --2- 172.21.15.62:0/690177403 >> v2:172.21.15.81:3300/0 conn(0x7f4be40b98b0 0x7f4be4161bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 >> v1:172.21.15.81:6789/0 conn(0x7f4be415e180 legacy=0x7f4be40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4162240 con 0x7f4be40d1020 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f4bd4001670 con 0x7f4be40d1020 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 >> v1:172.21.15.62:6789/0 conn(0x7f4be40d1020 legacy=0x7f4be40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 0x7f4bc8004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.242+0000 7f4be1600640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc80037a0 0x7f4bc8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be1600640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4bc8004280 0x7f4bc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800c200 con 0x7f4bc8004bf0 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800c3b0 con 0x7f4bc80037a0 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800c5b0 con 0x7f4bc8004280 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be3400640 1 --2- 172.21.15.62:0/690177403 >> v2:172.21.15.81:3300/0 conn(0x7f4be40b98b0 0x7f4be4161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be2000640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc80037a0 0x7f4bc8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f69b000b860 con 0x7f69c80b9170 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.243+0000 7f4be2a00640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 0x7f4bc8004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be3400640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4bc8004280 0x7f4bc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be2a00640 1 -- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4bc8004280 msgr2=0x7f4bc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.245+0000 7f69bf400640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 0x7f69a406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.245+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b008ac40 con 0x7f69c80b9170 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be2a00640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4bc8004280 0x7f4bc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be2a00640 1 -- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc80037a0 msgr2=0x7f4bc8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be2a00640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc80037a0 0x7f4bc8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.244+0000 7f4be2a00640 1 -- 172.21.15.62:0/690177403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8003520 con 0x7f4bc8004bf0 2024-03-28T17:59:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.246+0000 7f4be3400640 1 -- 172.21.15.62:0/690177403 reap_dead start 2024-03-28T17:59:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.246+0000 7f4be2a00640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 0x7f4bc8004ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0001820 tx=0x7f4bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878af9834bdd447 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.246+0000 7f69c6a00640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 0x7f69a406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.246+0000 7fb78ac00640 1 Processor -- start 2024-03-28T17:59:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.247+0000 7fb78ac00640 1 -- start start 2024-03-28T17:59:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.248+0000 7f69c6a00640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 0x7f69a406e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69c8146d30 tx=0x7f69b800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.248+0000 7f8812c00640 1 Processor -- start 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.249+0000 7fb78ac00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 0x7fb7840ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.249+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.249+0000 7fb78ac00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7840d24a0 con 0x7fb784071d10 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.249+0000 7fb78ac00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7840d3330 con 0x7fb784071930 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.249+0000 7fb78ac00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d2720 con 0x7fb7840b3310 2024-03-28T17:59:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.248+0000 7f8812c00640 1 -- start start 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f88040ead00 0x7f880404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8804053720 con 0x7f8804006ef0 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8804053a00 con 0x7f880404ffd0 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812c00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8804054330 con 0x7f88040ead00 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7fb783400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7fb783400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54260/0 (socket says 172.21.15.62:54260) 2024-03-28T17:59:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7fb783400640 1 -- 172.21.15.62:0/2835429430 learned_addr learned my addr 172.21.15.62:0/2835429430 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8804006ef0 0x7f88040382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56606/0 (socket says 172.21.15.62:56606) 2024-03-28T17:59:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8812200640 1 -- 172.21.15.62:0/46037999 learned_addr learned my addr 172.21.15.62:0/46037999 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7f8811800640 1 --2- 172.21.15.62:0/46037999 >> v2:172.21.15.81:3300/0 conn(0x7f88040ead00 0x7f880404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.250+0000 7fb783e00640 1 --2- 172.21.15.62:0/2835429430 >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 0x7fb7840ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb782a00640 1 -- 172.21.15.62:0/2835429430 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169464540 0 0) 0x7fb77c001490 con 0x7fb784071d10 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb782a00640 1 -- 172.21.15.62:0/2835429430 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb768003520 con 0x7fb784071d10 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb783400640 1 -- 172.21.15.62:0/2835429430 >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 msgr2=0x7fb7840ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb783400640 1 --2- 172.21.15.62:0/2835429430 >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 0x7fb7840ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb783400640 1 -- 172.21.15.62:0/2835429430 >> v1:172.21.15.81:6789/0 conn(0x7fb784071d10 legacy=0x7fb784162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7fb783400640 1 -- 172.21.15.62:0/2835429430 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840d2a00 con 0x7fb7840b3310 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.252+0000 7fb783e00640 1 --2- 172.21.15.62:0/2835429430 >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 0x7fb7840ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7f8811800640 1 -- 172.21.15.62:0/46037999 >> v1:172.21.15.81:6789/0 conn(0x7f880404ffd0 legacy=0x7f8804037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7f8811800640 1 -- 172.21.15.62:0/46037999 >> v1:172.21.15.62:6789/0 conn(0x7f8804006ef0 legacy=0x7f88040382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.251+0000 7f8811800640 1 -- 172.21.15.62:0/46037999 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804053ce0 con 0x7f88040ead00 2024-03-28T17:59:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0001470 con 0x7f4bc8004bf0 2024-03-28T17:59:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd00050d0 con 0x7f4bc8004bf0 2024-03-28T17:59:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f69b0059710 con 0x7f69c80b9170 2024-03-28T17:59:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7f8811800640 1 --2- 172.21.15.62:0/46037999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88040ead00 0x7f880404fa90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88000097a0 tx=0x7f880002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47cefb90ac90f986 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7fb783400640 1 --2- 172.21.15.62:0/2835429430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb770009870 tx=0x7fb77002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6eaa21c50caef17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f880be00640 1 -- 172.21.15.62:0/46037999 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800034070 con 0x7f88040ead00 2024-03-28T17:59:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f880be00640 1 -- 172.21.15.62:0/46037999 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8800030610 con 0x7f88040ead00 2024-03-28T17:59:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f880be00640 1 -- 172.21.15.62:0/46037999 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88000309e0 con 0x7f88040ead00 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.253+0000 7f4be1600640 1 -- 172.21.15.62:0/690177403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0007020 con 0x7f4bc8004bf0 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f4be9600640 1 -- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 msgr2=0x7f4bc8004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f4be9600640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 0x7f4bc8004ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0001820 tx=0x7f4bc8005830 comp rx=0 tx=0).stop 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f4be9600640 1 -- 172.21.15.62:0/690177403 shutdown_connections 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f4be9600640 1 --2- 172.21.15.62:0/690177403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc8004bf0 0x7f4bc8004ff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.254+0000 7f4be9600640 1 -- 172.21.15.62:0/690177403 >> 172.21.15.62:0/690177403 conn(0x7f4be4066dd0 msgr2=0x7f4be40671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f263ce00640 1 Processor -- start 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f8812c00640 1 -- 172.21.15.62:0/46037999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88040ead00 msgr2=0x7f880404fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f8812c00640 1 --2- 172.21.15.62:0/46037999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88040ead00 0x7f880404fa90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88000097a0 tx=0x7f880002b960 comp rx=0 tx=0).stop 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7fb782a00640 1 -- 172.21.15.62:0/2835429430 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002f830 con 0x7fb7840b3310 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7fb782a00640 1 -- 172.21.15.62:0/2835429430 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002f9d0 con 0x7fb7840b3310 2024-03-28T17:59:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7fb782a00640 1 -- 172.21.15.62:0/2835429430 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002fda0 con 0x7fb7840b3310 2024-03-28T17:59:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f4be9600640 1 -- 172.21.15.62:0/690177403 shutdown_connections 2024-03-28T17:59:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f263ce00640 1 -- start start 2024-03-28T17:59:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f8812c00640 1 -- 172.21.15.62:0/46037999 shutdown_connections 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f8812c00640 1 --2- 172.21.15.62:0/46037999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88040ead00 0x7f880404fa90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f8812c00640 1 -- 172.21.15.62:0/46037999 >> 172.21.15.62:0/46037999 conn(0x7f8804009fb0 msgr2=0x7f880400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f263ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26380b56f0 0x7f26380b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f263ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b6e60 con 0x7f26380b7ec0 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f263ce00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b7100 con 0x7f263815dd60 2024-03-28T17:59:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.256+0000 7f263ce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b6c30 con 0x7f26380b56f0 2024-03-28T17:59:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.255+0000 7f4be9600640 1 -- 172.21.15.62:0/690177403 wait complete. 2024-03-28T17:59:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 -- 172.21.15.62:0/2835429430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 msgr2=0x7fb7840ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2835429430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb770009870 tx=0x7fb77002ba10 comp rx=0 tx=0).stop 2024-03-28T17:59:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7f8812c00640 1 -- 172.21.15.62:0/46037999 shutdown_connections 2024-03-28T17:59:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7f8812c00640 1 -- 172.21.15.62:0/46037999 wait complete. 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 -- 172.21.15.62:0/2835429430 shutdown_connections 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2835429430 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb7840b3310 0x7fb7840ced20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 --2- 172.21.15.62:0/2835429430 >> v2:172.21.15.62:3300/0 conn(0x7fb784071930 0x7fb7840ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7fb78ac00640 1 -- 172.21.15.62:0/2835429430 >> 172.21.15.62:0/2835429430 conn(0x7fb7840d3ef0 msgr2=0x7fb7840d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7f2637e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f26380b7ec0 0x7f26380b82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56620/0 (socket says 172.21.15.62:56620) 2024-03-28T17:59:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.257+0000 7f2637e00640 1 -- 172.21.15.62:0/278274300 learned_addr learned my addr 172.21.15.62:0/278274300 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.258+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98753962 0 0) 0x7f26380b6e60 con 0x7f26380b7ec0 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.258+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c003520 con 0x7f26380b7ec0 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.258+0000 7f2636a00640 1 --2- 172.21.15.62:0/278274300 >> v2:172.21.15.62:3300/0 conn(0x7f26380b56f0 0x7f26380b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.258+0000 7fe765600640 1 Processor -- start 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549169967 0 0) 0x7f261c003520 con 0x7f26380b7ec0 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 >> v2:172.21.15.62:3300/0 conn(0x7f26380b56f0 msgr2=0x7f26380b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 --2- 172.21.15.62:0/278274300 >> v2:172.21.15.62:3300/0 conn(0x7f26380b56f0 0x7f26380b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 >> v1:172.21.15.81:6789/0 conn(0x7f263815dd60 legacy=0x7f26380b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380b73e0 con 0x7f26380b7ec0 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f26300019f0 con 0x7f26380b7ec0 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 >> v1:172.21.15.62:6789/0 conn(0x7f26380b7ec0 legacy=0x7f26380b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.258+0000 7fb78ac00640 1 -- 172.21.15.62:0/2835429430 shutdown_connections 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7fe765600640 1 -- start start 2024-03-28T17:59:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7fb78ac00640 1 -- 172.21.15.62:0/2835429430 wait complete. 2024-03-28T17:59:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f8812c00640 1 Processor -- start 2024-03-28T17:59:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.260+0000 7f8812c00640 1 -- start start 2024-03-28T17:59:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.260+0000 7f4be9600640 1 Processor -- start 2024-03-28T17:59:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.260+0000 7f4be9600640 1 -- start start 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261c0039b0 0x7f261c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f261c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f261c0051c0 0x7f261c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00c040 con 0x7f261c004800 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00c1f0 con 0x7f261c0039b0 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.259+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00c3f0 con 0x7f261c0051c0 2024-03-28T17:59:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f3b05600640 1 Processor -- start 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.260+0000 7f2636a00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f261c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.260+0000 7f2637400640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261c0039b0 0x7f261c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2636a00640 1 -- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f261c0051c0 msgr2=0x7f261c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2636a00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f261c0051c0 0x7f261c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2636a00640 1 -- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261c0039b0 msgr2=0x7f261c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2636a00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261c0039b0 0x7f261c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2636a00640 1 -- 172.21.15.62:0/278274300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380b73e0 con 0x7f261c004800 2024-03-28T17:59:01.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7fe765600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 0x7fe7580572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f2637e00640 1 -- 172.21.15.62:0/278274300 reap_dead start 2024-03-28T17:59:01.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7fb78ac00640 1 Processor -- start 2024-03-28T17:59:01.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f4be9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4be4157020 0x7f4be41607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be9600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 0x7f4be4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880404ffd0 0x7f8804035fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88040ead00 0x7f8804036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804054940 con 0x7f880404ffd0 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040547c0 con 0x7f8804006ef0 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8812c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880404f550 con 0x7f88040ead00 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7f8811800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f8811800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51466/0 (socket says 172.21.15.62:51466) 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f8811800640 1 -- 172.21.15.62:0/1933054550 learned_addr learned my addr 172.21.15.62:0/1933054550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f8812200640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88040ead00 0x7f8804036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7fe765600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe758054310 0x7fe75803c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7fe765600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580e36e0 con 0x7fe758053750 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7fe765600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe758057b40 con 0x7fe758054310 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.262+0000 7fe765600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe758057e20 con 0x7fe7580e3300 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.261+0000 7f3b05600640 1 -- start start 2024-03-28T17:59:01.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f54ff200640 1 Processor -- start 2024-03-28T17:59:01.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be9600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 0x7f4be4146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f54ff200640 1 -- start start 2024-03-28T17:59:01.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40d2490 con 0x7f4be4157020 2024-03-28T17:59:01.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7fb78ac00640 1 -- start start 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be9600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4162800 con 0x7f4be4157790 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be9600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4161bf0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f4be2000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 0x7f4be4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7fe75f400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe758053750 0x7fe758053c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33774/0 (socket says 172.21.15.62:33774) 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7fe75f400640 1 -- 172.21.15.62:0/488241995 learned_addr learned my addr 172.21.15.62:0/488241995 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:01.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7fe75fe00640 1 --2- 172.21.15.62:0/488241995 >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 0x7fe7580572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f4be2000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 0x7f4be4160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51478/0 (socket says 172.21.15.62:51478) 2024-03-28T17:59:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f4be3400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 0x7f4be4146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f4be2000640 1 -- 172.21.15.62:0/402695458 learned_addr learned my addr 172.21.15.62:0/402695458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f3b05600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b000717e0 0x7f3b00166660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f3b05600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00166cc0 con 0x7f3b00162250 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f3b05600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00070a20 con 0x7f3b00072860 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.263+0000 7f3b05600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000cdd70 con 0x7f3b000717e0 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f3afe000640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3b00072860 0x7f3b00165f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33784/0 (socket says 172.21.15.62:33784) 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f3afe000640 1 -- 172.21.15.62:0/1826840292 learned_addr learned my addr 172.21.15.62:0/1826840292 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f2636a00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f261c004c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2624014ee0 tx=0x7f261c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41152ebca70b4158 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7fe764c00640 1 --2- 172.21.15.62:0/488241995 >> v2:172.21.15.62:3300/0 conn(0x7fe758054310 0x7fe75803c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624045070 con 0x7f261c004800 2024-03-28T17:59:01.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3aff400640 1 --2- 172.21.15.62:0/1826840292 >> v2:172.21.15.81:3300/0 conn(0x7f3b000717e0 0x7f3b00166660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2624041620 con 0x7f261c004800 2024-03-28T17:59:01.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f2636000640 1 -- 172.21.15.62:0/278274300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624014b30 con 0x7f261c004800 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f8811800640 1 -- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88040ead00 msgr2=0x7f8804036510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f8811800640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88040ead00 0x7f8804036510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f8811800640 1 -- 172.21.15.62:0/1933054550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880404ffd0 msgr2=0x7f8804035fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f8811800640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880404ffd0 0x7f8804035fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.264+0000 7f8811800640 1 -- 172.21.15.62:0/1933054550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040344c0 con 0x7f8804006ef0 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422394 0 0) 0x7f3b00166cc0 con 0x7f3b00162250 2024-03-28T17:59:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae4003520 con 0x7f3b00162250 2024-03-28T17:59:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f4be3400640 1 -- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 msgr2=0x7f4be4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f4be3400640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 0x7f4be4160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f4be3400640 1 -- 172.21.15.62:0/402695458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4be4157020 msgr2=0x7f4be41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f4be3400640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4be4157020 0x7f4be41607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f4be3400640 1 -- 172.21.15.62:0/402695458 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40cbdc0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f8811800640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88000098c0 tx=0x7f88000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7fe75ea00640 1 -- 172.21.15.62:0/488241995 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750187853 0 0) 0x7fe7580e36e0 con 0x7fe758053750 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7fe75ea00640 1 -- 172.21.15.62:0/488241995 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe748003520 con 0x7fe758053750 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142513720 0 0) 0x7f3ae4003520 con 0x7f3b00162250 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb784071930 0x7fb7840b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb784071d10 0x7fb7841f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb78ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840b81a0 con 0x7fb7840b3930 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb78ac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d2fc0 con 0x7fb784071d10 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb78ac00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d4dd0 con 0x7fb784071930 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f4be3400640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 0x7f4be4146a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4be40d5020 tx=0x7f4bdc00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00a460 con 0x7f4be40cd8e0 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00a600 con 0x7f4be40cd8e0 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00a9b0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800034040 con 0x7f8804006ef0 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880003acc0 con 0x7f8804006ef0 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 >> v2:172.21.15.81:3300/0 conn(0x7f3b000717e0 msgr2=0x7f3b00166660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 --2- 172.21.15.62:0/1826840292 >> v2:172.21.15.81:3300/0 conn(0x7f3b000717e0 0x7f3b00166660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 >> v1:172.21.15.81:6789/0 conn(0x7f3b00072860 legacy=0x7f3b00165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00070d00 con 0x7f3b00162250 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f3af0001670 con 0x7f3b00162250 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 >> v1:172.21.15.62:6789/0 conn(0x7f3b00162250 legacy=0x7f3b00072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3ae40039b0 0x7f3ae4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae4004800 0x7f3ae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3ae400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000cdd70 con 0x7f3ae4004800 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae400c060 con 0x7f3ae40051c0 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae400c260 con 0x7f3ae40039b0 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f3aff400640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3ae400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f54ff200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54f80b3f60 0x7f54f8071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.265+0000 7f54ff200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f54f815e300 0x7f54f80b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54ff200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f80ae8e0 con 0x7f54f815df20 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54ff200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f8071a30 con 0x7f54f815e300 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54ff200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f80ae040 con 0x7f54f80b3f60 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54fd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f54f815e300 0x7f54f80b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54fd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f54f815e300 0x7f54f80b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47036/0 (socket says 172.21.15.62:47036) 2024-03-28T17:59:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7f54fd200640 1 -- 172.21.15.62:0/650951803 learned_addr learned my addr 172.21.15.62:0/650951803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54f7400640 1 --2- 172.21.15.62:0/650951803 >> v2:172.21.15.81:3300/0 conn(0x7f54f80b3f60 0x7f54f8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54f6a00640 1 -- 172.21.15.62:0/650951803 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332170051 0 0) 0x7f54f80ae8e0 con 0x7f54f815df20 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54f6a00640 1 -- 172.21.15.62:0/650951803 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54dc003520 con 0x7f54f815df20 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb788c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb788c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47064/0 (socket says 172.21.15.62:47064) 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7fb788c00640 1 -- 172.21.15.62:0/3757266278 learned_addr learned my addr 172.21.15.62:0/3757266278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f263ce00640 1 -- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 msgr2=0x7f261c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7fe764c00640 1 -- 172.21.15.62:0/488241995 >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 msgr2=0x7fe7580572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7fe764c00640 1 --2- 172.21.15.62:0/488241995 >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 0x7fe7580572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7fe764c00640 1 -- 172.21.15.62:0/488241995 >> v1:172.21.15.81:6789/0 conn(0x7fe758053750 legacy=0x7fe758053c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.266+0000 7fe764c00640 1 -- 172.21.15.62:0/488241995 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758058100 con 0x7fe758054310 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb783e00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb784071930 0x7fb7840b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb783400640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb784071d10 0x7fb7841f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be40cc0a0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f263ce00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f261c004c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2624014ee0 tx=0x7f261c005830 comp rx=0 tx=0).stop 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3afe000640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae4004800 0x7f3ae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3aff400640 1 -- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3ae40039b0 msgr2=0x7f3ae4003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3aff400640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3ae40039b0 0x7f3ae4003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3aff400640 1 -- 172.21.15.62:0/1826840292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae4004800 msgr2=0x7f3ae4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88040347a0 con 0x7f8804006ef0 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8804034d60 con 0x7f8804006ef0 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb788c00640 1 -- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb784071930 msgr2=0x7fb7840b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb788c00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb784071930 0x7fb7840b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb788c00640 1 -- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb784071d10 msgr2=0x7fb7841f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb788c00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb784071d10 0x7fb7841f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7fb788c00640 1 -- 172.21.15.62:0/3757266278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7841f4d30 con 0x7fb7840b3930 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be40cc660 con 0x7f4be40cd8e0 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54fd200640 1 -- 172.21.15.62:0/650951803 >> v2:172.21.15.81:3300/0 conn(0x7f54f80b3f60 msgr2=0x7f54f8071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54fd200640 1 --2- 172.21.15.62:0/650951803 >> v2:172.21.15.81:3300/0 conn(0x7f54f80b3f60 0x7f54f8071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54fd200640 1 -- 172.21.15.62:0/650951803 >> v1:172.21.15.62:6789/0 conn(0x7f54f815df20 legacy=0x7f54f80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.267+0000 7f54fd200640 1 -- 172.21.15.62:0/650951803 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80ae320 con 0x7f54f815e300 2024-03-28T17:59:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800044630 con 0x7f8804006ef0 2024-03-28T17:59:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.269+0000 7fe75fe00640 1 --2- 172.21.15.62:0/488241995 >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 0x7fe7580572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.269+0000 7f54fd200640 1 --2- 172.21.15.62:0/650951803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f815e300 0x7f54f80b2cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54f806ea60 tx=0x7f54f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f5adc9dcf4aba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e8005290 con 0x7f8804006ef0 2024-03-28T17:59:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3aff400640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae4004800 0x7f3ae4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.268+0000 7f3aff400640 1 -- 172.21.15.62:0/1826840292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00070d00 con 0x7f3ae40051c0 2024-03-28T17:59:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f263ce00640 1 -- 172.21.15.62:0/278274300 shutdown_connections 2024-03-28T17:59:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f3afe000640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae4004800 0x7f3ae4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f263ce00640 1 --2- 172.21.15.62:0/278274300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f261c004c80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f263ce00640 1 -- 172.21.15.62:0/278274300 >> 172.21.15.62:0/278274300 conn(0x7f2638071ae0 msgr2=0x7f2638071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.269+0000 7fb788c00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001630 tx=0x7fb77c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003b70 con 0x7fb7840b3930 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c003d10 con 0x7fb7840b3930 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00aa70 con 0x7fb7840b3930 2024-03-28T17:59:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.270+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4073f20 con 0x7f4be40cd8e0 2024-03-28T17:59:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7f3aff400640 1 -- 172.21.15.62:0/1826840292 reap_dead start 2024-03-28T17:59:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78415f860 con 0x7fb7840b3930 2024-03-28T17:59:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7f263ce00640 1 -- 172.21.15.62:0/278274300 shutdown_connections 2024-03-28T17:59:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7f263ce00640 1 -- 172.21.15.62:0/278274300 wait complete. 2024-03-28T17:59:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7f3aff400640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3ae400b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b000af670 tx=0x7f3ae4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b86df2b793022f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7f54f6a00640 1 -- 172.21.15.62:0/650951803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f002ca80 con 0x7f54f815e300 2024-03-28T17:59:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7f54f6a00640 1 -- 172.21.15.62:0/650951803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f002cc20 con 0x7f54f815e300 2024-03-28T17:59:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.271+0000 7f54f6a00640 1 -- 172.21.15.62:0/650951803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00343f0 con 0x7f54f815e300 2024-03-28T17:59:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7840b41b0 con 0x7fb7840b3930 2024-03-28T17:59:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7fe764c00640 1 --2- 172.21.15.62:0/488241995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758054310 0x7fe75803c120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7580572c0 tx=0x7fe75402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b57dfa13eea13d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8039070 con 0x7f3ae40051c0 2024-03-28T17:59:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af802cbd0 con 0x7f3ae40051c0 2024-03-28T17:59:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f3afd600640 1 -- 172.21.15.62:0/1826840292 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8009c10 con 0x7f3ae40051c0 2024-03-28T17:59:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f880004b060 con 0x7f8804006ef0 2024-03-28T17:59:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7840680c0 con 0x7fb7840b3930 2024-03-28T17:59:01.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 -- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 msgr2=0x7f3ae400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3ae400b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b000af670 tx=0x7f3ae4005830 comp rx=0 tx=0).stop 2024-03-28T17:59:01.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4bdc005d10 con 0x7f4be40cd8e0 2024-03-28T17:59:01.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f4bdaa00640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 0x7f4bc406e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bdc08b3e0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 -- 172.21.15.62:0/1826840292 shutdown_connections 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 --2- 172.21.15.62:0/1826840292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3ae400b9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 -- 172.21.15.62:0/1826840292 >> 172.21.15.62:0/1826840292 conn(0x7f3b000b1730 msgr2=0x7f3b000b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7f54ff200640 1 -- 172.21.15.62:0/650951803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f815e300 msgr2=0x7f54f80b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.272+0000 7f54ff200640 1 --2- 172.21.15.62:0/650951803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f815e300 0x7f54f80b2cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54f806ea60 tx=0x7f54f002bee0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f54ff200640 1 -- 172.21.15.62:0/650951803 shutdown_connections 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f54ff200640 1 --2- 172.21.15.62:0/650951803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f815e300 0x7f54f80b2cb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f54ff200640 1 --2- 172.21.15.62:0/650951803 >> v2:172.21.15.81:3300/0 conn(0x7f54f80b3f60 0x7f54f8071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.273+0000 7f54ff200640 1 -- 172.21.15.62:0/650951803 >> 172.21.15.62:0/650951803 conn(0x7f54f8066dd0 msgr2=0x7f54f80671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f263ce00640 1 Processor -- start 2024-03-28T17:59:01.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f4be2a00640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 0x7f4bc406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f54ff200640 1 -- 172.21.15.62:0/650951803 shutdown_connections 2024-03-28T17:59:01.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7fe75ea00640 1 -- 172.21.15.62:0/488241995 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540315a0 con 0x7fe758054310 2024-03-28T17:59:01.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 -- 172.21.15.62:0/1826840292 shutdown_connections 2024-03-28T17:59:01.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f3b05600640 1 -- 172.21.15.62:0/1826840292 wait complete. 2024-03-28T17:59:01.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f263ce00640 1 -- start start 2024-03-28T17:59:01.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.275+0000 7f54ff200640 1 -- 172.21.15.62:0/650951803 wait complete. 2024-03-28T17:59:01.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7f8809600640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 0x7f87ec06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88000ac960 con 0x7f8804006ef0 2024-03-28T17:59:01.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb77c00ac10 con 0x7fb7840b3930 2024-03-28T17:59:01.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7f8810e00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 0x7f87ec06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7f263ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.276+0000 7f263ce00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2638155ea0 0x7f2638156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f263ce00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2638157170 0x7f263815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f263ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263815e140 con 0x7f261c004800 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f263ce00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b79a0 con 0x7f2638157170 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f263ce00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b6370 con 0x7f2638155ea0 2024-03-28T17:59:01.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f4be2a00640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 0x7f4bc406e190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0033000 tx=0x7f4bd0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f3b05600640 1 Processor -- start 2024-03-28T17:59:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f2637400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f2637400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47080/0 (socket says 172.21.15.62:47080) 2024-03-28T17:59:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f2637400640 1 -- 172.21.15.62:0/3337284323 learned_addr learned my addr 172.21.15.62:0/3337284323 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.277+0000 7f3b05600640 1 -- start start 2024-03-28T17:59:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2636a00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2638155ea0 0x7f2638156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7fb77be00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 0x7fb76406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f54ff200640 1 Processor -- start 2024-03-28T17:59:01.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637e00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2638157170 0x7f263815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637400640 1 -- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2638155ea0 msgr2=0x7f2638156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637400640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2638155ea0 0x7f2638156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637400640 1 -- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2638157170 msgr2=0x7f263815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f54ff200640 1 -- start start 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637400640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2638157170 0x7f263815ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7f2637400640 1 -- 172.21.15.62:0/3337284323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380c9b20 con 0x7f261c004800 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.278+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb77c08b300 con 0x7fb7840b3930 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f2637400640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262800e7b0 tx=0x7f262800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3b0015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 0x7f3b001660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00162630 con 0x7f3b0015bf90 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000712c0 con 0x7f3ae40051c0 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f3b05600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001627b0 con 0x7f3b00163fe0 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7fb783e00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 0x7fb76406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.279+0000 7f8810e00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 0x7f87ec06e0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000f60 tx=0x7f87f80079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f3aff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f3aff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47090/0 (socket says 172.21.15.62:47090) 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f3aff400640 1 -- 172.21.15.62:0/1984957662 learned_addr learned my addr 172.21.15.62:0/1984957662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f54f815df20 0x7f54f81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 0x7f54f809cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f806d030 con 0x7f54f80b3f60 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80af6b0 con 0x7f54f815e300 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54ff200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f806d1b0 con 0x7f54f815df20 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f3afea00640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3b0015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47094/0 (socket says 172.21.15.62:47094) 2024-03-28T17:59:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f54f7e00640 1 -- 172.21.15.62:0/608166465 learned_addr learned my addr 172.21.15.62:0/608166465 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7fb783e00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 0x7fb76406e0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb774009560 tx=0x7fb774009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.280+0000 7f3aff400640 1 -- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 msgr2=0x7f3b001660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3afe000640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 0x7f3b001660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7400640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f54f815df20 0x7f54f81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3aff400640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 0x7f3b001660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3aff400640 1 -- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 msgr2=0x7f3b0015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3aff400640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3b0015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54fd200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 0x7f54f809cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3aff400640 1 -- 172.21.15.62:0/1984957662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0015a6b0 con 0x7f3b0015bf90 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 -- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f54f815df20 msgr2=0x7f54f81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f54f815df20 0x7f54f81567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 -- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 msgr2=0x7f54f809cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 0x7f54f809cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 -- 172.21.15.62:0/608166465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f809b360 con 0x7f54f80b3f60 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54fd200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 0x7f54f809cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f3afe000640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 0x7f3b001660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.281+0000 7f54f7e00640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c4a0 tx=0x7f54e800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.282+0000 7f3aff400640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3af8034d40 tx=0x7f3af8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280050d0 con 0x7f261c004800 2024-03-28T17:59:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628018070 con 0x7f261c004800 2024-03-28T17:59:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800c6c0 con 0x7f261c004800 2024-03-28T17:59:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7fe75ea00640 1 -- 172.21.15.62:0/488241995 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754037070 con 0x7fe758054310 2024-03-28T17:59:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7fe75ea00640 1 -- 172.21.15.62:0/488241995 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754031940 con 0x7fe758054310 2024-03-28T17:59:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380c9e00 con 0x7f261c004800 2024-03-28T17:59:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8017070 con 0x7f54f80b3f60 2024-03-28T17:59:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8044020 con 0x7f3b0015bf90 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8005480 con 0x7f54f80b3f60 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f809b640 con 0x7f54f80b3f60 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003830 con 0x7f54f80b3f60 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f809bc00 con 0x7f54f80b3f60 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af803f470 con 0x7f3b0015bf90 2024-03-28T17:59:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.284+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8039040 con 0x7f3b0015bf90 2024-03-28T17:59:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.283+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2638155b40 con 0x7f261c004800 2024-03-28T17:59:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0015a990 con 0x7f3b0015bf90 2024-03-28T17:59:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2638072b90 con 0x7f261c004800 2024-03-28T17:59:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7fe765600640 1 -- 172.21.15.62:0/488241995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758054310 msgr2=0x7fe75803c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7fe765600640 1 --2- 172.21.15.62:0/488241995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758054310 0x7fe75803c120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7580572c0 tx=0x7fe75402b840 comp rx=0 tx=0).stop 2024-03-28T17:59:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb7841f4d30 con 0x7fb7840b3930 2024-03-28T17:59:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7fe765600640 1 -- 172.21.15.62:0/488241995 shutdown_connections 2024-03-28T17:59:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.285+0000 7fe765600640 1 --2- 172.21.15.62:0/488241995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758054310 0x7fe75803c120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.286+0000 7fe765600640 1 --2- 172.21.15.62:0/488241995 >> v2:172.21.15.81:3300/0 conn(0x7fe7580e3300 0x7fe7580572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.286+0000 7fe765600640 1 -- 172.21.15.62:0/488241995 >> 172.21.15.62:0/488241995 conn(0x7fe758058ef0 msgr2=0x7fe758059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.286+0000 7fe765600640 1 -- 172.21.15.62:0/488241995 shutdown_connections 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.286+0000 7fe765600640 1 -- 172.21.15.62:0/488241995 wait complete. 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.286+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0015bbe0 con 0x7f3b0015bf90 2024-03-28T17:59:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.288+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d0005290 con 0x7f54f80b3f60 2024-03-28T17:59:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.288+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0004f470 con 0x7f3b0015bf90 2024-03-28T17:59:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.288+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3af8047020 con 0x7f3b0015bf90 2024-03-28T17:59:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7f3af6a00640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 0x7f3ae006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3af80acde0 con 0x7f3b0015bf90 2024-03-28T17:59:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7fe765600640 1 Processor -- start 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4be4073f20 con 0x7f4be40cd8e0 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.288+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f262800c860 con 0x7f261c004800 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7f262f400640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 0x7f261806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.289+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f262808a760 con 0x7f261c004800 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.290+0000 7fe765600640 1 -- start start 2024-03-28T17:59:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.288+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f54e801a070 con 0x7f54f80b3f60 2024-03-28T17:59:01.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.290+0000 7f3afea00640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 0x7f3ae006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.290+0000 7f54efe00640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 0x7f54d806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.290+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54e8089dd0 con 0x7f54f80b3f60 2024-03-28T17:59:01.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.290+0000 7f2636a00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 0x7f261806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe758054310 0x7fe7580e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7580e3300 0x7fe7580dd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758058be0 con 0x7fe758053750 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758057830 con 0x7fe7580e3300 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe765600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580d9710 con 0x7fe758054310 2024-03-28T17:59:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7f54f7400640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 0x7f54d806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f880003f070 con 0x7f8804006ef0 2024-03-28T17:59:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.291+0000 7fe75fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47104/0 (socket says 172.21.15.62:47104) 2024-03-28T17:59:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 -- 172.21.15.62:0/2497725758 learned_addr learned my addr 172.21.15.62:0/2497725758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75f400640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe758054310 0x7fe7580e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7f2636a00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 0x7f261806e090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2624047000 tx=0x7f2624008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7f3afea00640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 0x7f3ae006e110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3af0001eb0 tx=0x7f3af0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe764c00640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7580e3300 0x7fe7580dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 -- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe758054310 msgr2=0x7fe7580e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe758054310 0x7fe7580e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 -- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7580e3300 msgr2=0x7fe7580dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7580e3300 0x7fe7580dd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.292+0000 7fe75fe00640 1 -- 172.21.15.62:0/2497725758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758051a90 con 0x7fe758053750 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.293+0000 7fe75fe00640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7600697d0 tx=0x7fe760073610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.293+0000 7f54f7400640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 0x7f54d806e090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54f8157590 tx=0x7f54e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.295+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f26280591b0 con 0x7f261c004800 2024-03-28T17:59:01.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.300+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760095870 con 0x7fe758053750 2024-03-28T17:59:01.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.301+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f54e808d030 con 0x7f54f80b3f60 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.301+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe758051d70 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.301+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75804d750 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.303+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75804d2a0 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.309+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760099070 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.309+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760095c10 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.309+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe76009e470 con 0x7fe758053750 2024-03-28T17:59:01.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.309+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3af807b360 con 0x7f3b0015bf90 2024-03-28T17:59:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.311+0000 7fe753e00640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 0x7fe74006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.311+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe76010e7d0 con 0x7fe758053750 2024-03-28T17:59:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.313+0000 7fe75f400640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 0x7fe74006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.316+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe7600ad080 con 0x7fe758053750 2024-03-28T17:59:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.316+0000 7fe75f400640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 0x7fe74006e0f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe75817c6a0 tx=0x7fe74c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:01.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.686+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f69a0005580 con 0x7f69c80b9170 2024-03-28T17:59:01.687 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.687+0000 7f69bf400640 1 -- 172.21.15.62:0/3974068696 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69b0059010 con 0x7f69c80b9170 2024-03-28T17:59:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.690+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 msgr2=0x7f69a406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.690+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 0x7f69a406e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69c8146d30 tx=0x7f69b800b040 comp rx=0 tx=0).stop 2024-03-28T17:59:01.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.691+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 msgr2=0x7f69c8164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.691+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d7d0 tx=0x7f69b000dc90 comp rx=0 tx=0).stop 2024-03-28T17:59:01.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.691+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 shutdown_connections 2024-03-28T17:59:01.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f69a406bca0 0x7f69a406e160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f69c8165890 0x7f69c8160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69c80b96e0 0x7f69c8146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 --2- 172.21.15.62:0/3974068696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69c80b9170 0x7f69c8164980 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 >> 172.21.15.62:0/3974068696 conn(0x7f69c8071cf0 msgr2=0x7f69c80b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.692+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 shutdown_connections 2024-03-28T17:59:01.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.693+0000 7f69cd200640 1 -- 172.21.15.62:0/3974068696 wait complete. 2024-03-28T17:59:01.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-03-28T17:59:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.728+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2638157550 con 0x7f261c004800 2024-03-28T17:59:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.728+0000 7f262f400640 1 -- 172.21.15.62:0/3337284323 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2628058ab0 con 0x7f261c004800 2024-03-28T17:59:01.729 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:01.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.730+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7840b9710 con 0x7fb7840b3930 2024-03-28T17:59:01.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.731+0000 7fb77be00640 1 -- 172.21.15.62:0/3757266278 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7840d24a0 con 0x7fb7840b3930 2024-03-28T17:59:01.731 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 msgr2=0x7f261806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 0x7f261806e090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2624047000 tx=0x7f2624008030 comp rx=0 tx=0).stop 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 msgr2=0x7f2638155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262800e7b0 tx=0x7f262800ec70 comp rx=0 tx=0).stop 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 shutdown_connections 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f261806bbd0 0x7f261806e090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2638157170 0x7f263815ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2638155ea0 0x7f2638156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 --2- 172.21.15.62:0/3337284323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f261c004800 0x7f2638155600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 >> 172.21.15.62:0/3337284323 conn(0x7f2638071ae0 msgr2=0x7f26380d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 shutdown_connections 2024-03-28T17:59:01.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.732+0000 7f263ce00640 1 -- 172.21.15.62:0/3337284323 wait complete. 2024-03-28T17:59:01.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.733+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4be40687d0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.734+0000 7f4bdaa00640 1 -- 172.21.15.62:0/402695458 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4bdc059eb0 con 0x7f4be40cd8e0 2024-03-28T17:59:01.734 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2024-03-28T17:59:01.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 msgr2=0x7fb76406e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 0x7fb76406e0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb774009560 tx=0x7fb774009190 comp rx=0 tx=0).stop 2024-03-28T17:59:01.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 msgr2=0x7fb78415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001630 tx=0x7fb77c00cbe0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 msgr2=0x7f4bc406e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 0x7f4bc406e190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0033000 tx=0x7f4bd0030000 comp rx=0 tx=0).stop 2024-03-28T17:59:01.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 shutdown_connections 2024-03-28T17:59:01.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb76406bc20 0x7fb76406e0e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7840b3930 0x7fb78415f320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb784071d10 0x7fb7841f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 --2- 172.21.15.62:0/3757266278 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb784071930 0x7fb7840b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.737+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 >> 172.21.15.62:0/3757266278 conn(0x7fb7840d3ef0 msgr2=0x7fb78415d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 shutdown_connections 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 msgr2=0x7f4be4146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 0x7f4be4146a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4be40d5020 tx=0x7f4bdc00c810 comp rx=0 tx=0).stop 2024-03-28T17:59:01.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7fb78ac00640 1 -- 172.21.15.62:0/3757266278 wait complete. 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 shutdown_connections 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4bc406bcd0 0x7f4bc406e190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4be40cd8e0 0x7f4be4146a00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4be4157790 0x7f4be4160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 --2- 172.21.15.62:0/402695458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4be4157020 0x7f4be41607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 >> 172.21.15.62:0/402695458 conn(0x7f4be4066dd0 msgr2=0x7f4be40b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 shutdown_connections 2024-03-28T17:59:01.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.738+0000 7f4be9600640 1 -- 172.21.15.62:0/402695458 wait complete. 2024-03-28T17:59:01.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.741+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f87e8005580 con 0x7f8804006ef0 2024-03-28T17:59:01.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.742+0000 7f8809600640 1 -- 172.21.15.62:0/1933054550 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f880007b3b0 con 0x7f8804006ef0 2024-03-28T17:59:01.743 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:01.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 msgr2=0x7f87ec06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 0x7f87ec06e0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000f60 tx=0x7f87f80079b0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 msgr2=0x7f88040dd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88000098c0 tx=0x7f88000046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 shutdown_connections 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f87ec06bc20 0x7f87ec06e0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f88040ead00 0x7f8804036510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880404ffd0 0x7f8804035fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 --2- 172.21.15.62:0/1933054550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8804006ef0 0x7f88040dd2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.746+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 >> 172.21.15.62:0/1933054550 conn(0x7f8804009fb0 msgr2=0x7f880404dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.747+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 shutdown_connections 2024-03-28T17:59:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.747+0000 7f8812c00640 1 -- 172.21.15.62:0/1933054550 wait complete. 2024-03-28T17:59:01.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-03-28T17:59:01.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.752+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f54d0005580 con 0x7f54f80b3f60 2024-03-28T17:59:01.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.753+0000 7f54efe00640 1 -- 172.21.15.62:0/608166465 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54e80581d0 con 0x7f54f80b3f60 2024-03-28T17:59:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:01.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.755+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe758005710 con 0x7fe758053750 2024-03-28T17:59:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7fe753e00640 1 -- 172.21.15.62:0/2497725758 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe760071050 con 0x7fe758053750 2024-03-28T17:59:01.756 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 msgr2=0x7f54d806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 0x7f54d806e090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54f8157590 tx=0x7f54e4008040 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 msgr2=0x7f54f809aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c4a0 tx=0x7f54e800c960 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 shutdown_connections 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f54d806bbd0 0x7f54d806e090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f815e300 0x7f54f809cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f54f815df20 0x7f54f81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 --2- 172.21.15.62:0/608166465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3f60 0x7f54f809aa60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.756+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 >> 172.21.15.62:0/608166465 conn(0x7f54f8066dd0 msgr2=0x7f54f80671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.757+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 shutdown_connections 2024-03-28T17:59:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.757+0000 7f54ff200640 1 -- 172.21.15.62:0/608166465 wait complete. 2024-03-28T17:59:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 msgr2=0x7fe74006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 0x7fe74006e0f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe75817c6a0 tx=0x7fe74c009040 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 msgr2=0x7fe7580d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7600697d0 tx=0x7fe760073610 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 shutdown_connections 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe74006bc30 0x7fe74006e0f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7580e3300 0x7fe7580dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe758054310 0x7fe7580e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 --2- 172.21.15.62:0/2497725758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe758053750 0x7fe7580d91d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 >> 172.21.15.62:0/2497725758 conn(0x7fe758058ef0 msgr2=0x7fe7580e25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.760+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 shutdown_connections 2024-03-28T17:59:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.760+0000 7fe765600640 1 -- 172.21.15.62:0/2497725758 wait complete. 2024-03-28T17:59:01.761 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2024-03-28T17:59:01.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.759+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3b000b2360 con 0x7f3b0015bf90 2024-03-28T17:59:01.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.760+0000 7f3af6a00640 1 -- 172.21.15.62:0/1984957662 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3af807f400 con 0x7f3b0015bf90 2024-03-28T17:59:01.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.763+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 msgr2=0x7f3ae006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.763+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 0x7f3ae006e110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3af0001eb0 tx=0x7f3af0009040 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.763+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 msgr2=0x7f3b00153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.763+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3af8034d40 tx=0x7f3af8009750 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 shutdown_connections 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3ae006bc50 0x7f3ae006e110 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0015bf90 0x7f3b00153790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3b00163fe0 0x7f3b001660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 --2- 172.21.15.62:0/1984957662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ae40051c0 0x7f3b0015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 >> 172.21.15.62:0/1984957662 conn(0x7f3b000b1730 msgr2=0x7f3b0006d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 shutdown_connections 2024-03-28T17:59:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:01.764+0000 7f3b05600640 1 -- 172.21.15.62:0/1984957662 wait complete. 2024-03-28T17:59:01.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-03-28T17:59:01.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-03-28T17:59:01.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-03-28T17:59:01.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-03-28T17:59:01.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-03-28T17:59:01.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-03-28T17:59:02.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T17:59:02.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T17:59:02.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T17:59:02.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T17:59:02.772 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T17:59:02.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T17:59:02.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T17:59:02.786 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-28T17:59:02.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.996+0000 7fbd86600640 1 Processor -- start 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.997+0000 7fbd86600640 1 -- start start 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.999+0000 7fbd86600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd800b3310 0x7fbd800ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.999+0000 7fbd86600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071d10 0x7fbd800b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.999+0000 7fbd86600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800d24a0 con 0x7fbd80071930 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.999+0000 7fbd86600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800d3330 con 0x7fbd800b3310 2024-03-28T17:59:02.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:02.999+0000 7fbd86600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800d2720 con 0x7fbd80071d10 2024-03-28T17:59:03.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7ea00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd800b3310 0x7fbd800ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7ea00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd800b3310 0x7fbd800ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51554/0 (socket says 172.21.15.62:51554) 2024-03-28T17:59:03.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7ea00640 1 -- 172.21.15.62:0/799988974 learned_addr learned my addr 172.21.15.62:0/799988974 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7fe00640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071d10 0x7fbd800b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7ea00640 1 -- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071d10 msgr2=0x7fbd800b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.000+0000 7fbd7ea00640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071d10 0x7fbd800b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.001+0000 7fbd7ea00640 1 -- 172.21.15.62:0/799988974 >> v1:172.21.15.81:6789/0 conn(0x7fbd80071930 legacy=0x7fbd800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.001+0000 7fbd7ea00640 1 -- 172.21.15.62:0/799988974 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800d2a00 con 0x7fbd800b3310 2024-03-28T17:59:03.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.002+0000 7fbd7ea00640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd800b3310 0x7fbd800ceef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de4261ee37648de server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.003+0000 7fbd7e000640 1 -- 172.21.15.62:0/799988974 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7002f870 con 0x7fbd800b3310 2024-03-28T17:59:03.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.003+0000 7fbd7e000640 1 -- 172.21.15.62:0/799988974 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7002fa10 con 0x7fbd800b3310 2024-03-28T17:59:03.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.003+0000 7fbd7e000640 1 -- 172.21.15.62:0/799988974 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7002fe20 con 0x7fbd800b3310 2024-03-28T17:59:03.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.004+0000 7fbd86600640 1 -- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd800b3310 msgr2=0x7fbd800ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.004+0000 7fbd86600640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd800b3310 0x7fbd800ceef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002ba50 comp rx=0 tx=0).stop 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 -- 172.21.15.62:0/799988974 shutdown_connections 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071d10 0x7fbd800b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 --2- 172.21.15.62:0/799988974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd800b3310 0x7fbd800ceef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 -- 172.21.15.62:0/799988974 >> 172.21.15.62:0/799988974 conn(0x7fbd800d3ef0 msgr2=0x7fbd800d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 -- 172.21.15.62:0/799988974 shutdown_connections 2024-03-28T17:59:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.005+0000 7fbd86600640 1 -- 172.21.15.62:0/799988974 wait complete. 2024-03-28T17:59:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.010+0000 7fbd86600640 1 Processor -- start 2024-03-28T17:59:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.010+0000 7fbd86600640 1 -- start start 2024-03-28T17:59:03.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.012+0000 7fbd86600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071930 0x7fbd801614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.013+0000 7fbd86600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd80071d10 0x7fbd800708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.013+0000 7fbd86600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd86600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800d4dd0 con 0x7fbd800b3310 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd86600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800d2fc0 con 0x7fbd80071d10 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47118/0 (socket says 172.21.15.62:47118) 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd7fe00640 1 -- 172.21.15.62:0/226362894 learned_addr learned my addr 172.21.15.62:0/226362894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.014+0000 7fbd86600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800d37c0 con 0x7fbd80071930 2024-03-28T17:59:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.015+0000 7fbd7fe00640 1 -- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071930 msgr2=0x7fbd801614f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T17:59:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.015+0000 7fbd7fe00640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071930 0x7fbd801614f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.015+0000 7fbd7fe00640 1 -- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd80071d10 msgr2=0x7fbd800708c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T17:59:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.015+0000 7fbd7fe00640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd80071d10 0x7fbd800708c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.015+0000 7fbd7fe00640 1 -- 172.21.15.62:0/226362894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd801fe1c0 con 0x7fbd800b3310 2024-03-28T17:59:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.016+0000 7fbd7fe00640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e850 tx=0x7fbd6c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.017+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c017070 con 0x7fbd800b3310 2024-03-28T17:59:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.017+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00c8e0 con 0x7fbd800b3310 2024-03-28T17:59:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.017+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00cc90 con 0x7fbd800b3310 2024-03-28T17:59:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.017+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd802017e0 con 0x7fbd800b3310 2024-03-28T17:59:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.017+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd80201da0 con 0x7fbd800b3310 2024-03-28T17:59:03.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.019+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd800680c0 con 0x7fbd800b3310 2024-03-28T17:59:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.022+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd6c01a070 con 0x7fbd800b3310 2024-03-28T17:59:03.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.025+0000 7fbd77400640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 0x7fbd6006e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.025+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd6c08a9a0 con 0x7fbd800b3310 2024-03-28T17:59:03.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.026+0000 7fbd7f400640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 0x7fbd6006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.028+0000 7fbd7f400640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 0x7fbd6006e190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd800d3ec0 tx=0x7fbd78007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.033+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbd6c059500 con 0x7fbd800b3310 2024-03-28T17:59:03.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.037+0000 7f30d6000640 1 Processor -- start 2024-03-28T17:59:03.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.037+0000 7f30d6000640 1 -- start start 2024-03-28T17:59:03.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.038+0000 7f30d6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30d00b56f0 0x7f30d00b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.038+0000 7f30d6000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00b8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30d6000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d00b6e60 con 0x7f30d015dd60 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30d6000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d00b6c30 con 0x7f30d00b56f0 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30d6000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b6090 con 0x7f30d00b7ec0 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30cfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30d00b56f0 0x7f30d00b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30cfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30d00b56f0 0x7f30d00b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47132/0 (socket says 172.21.15.62:47132) 2024-03-28T17:59:03.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30d4c00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f30d015dd60 0x7f30d00b94b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33794/0 (socket says 172.21.15.62:33794) 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30cfe00640 1 -- 172.21.15.62:0/3785959077 learned_addr learned my addr 172.21.15.62:0/3785959077 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.039+0000 7f30d5600640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.040+0000 7f30cfe00640 1 -- 172.21.15.62:0/3785959077 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 msgr2=0x7f30d00b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.040+0000 7f30cfe00640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00b8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.040+0000 7f30cfe00640 1 -- 172.21.15.62:0/3785959077 >> v1:172.21.15.81:6789/0 conn(0x7f30d015dd60 legacy=0x7f30d00b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.040+0000 7f30cfe00640 1 -- 172.21.15.62:0/3785959077 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d00b7100 con 0x7f30d00b56f0 2024-03-28T17:59:03.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.042+0000 7f8d29a00640 1 Processor -- start 2024-03-28T17:59:03.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.042+0000 7f8d29a00640 1 -- start start 2024-03-28T17:59:03.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.042+0000 7f30cfe00640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00b5b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009870 tx=0x7f30bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a58ae83bef1c3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.042+0000 7f30cf400640 1 -- 172.21.15.62:0/3785959077 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034070 con 0x7f30d00b56f0 2024-03-28T17:59:03.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.043+0000 7f30cf400640 1 -- 172.21.15.62:0/3785959077 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc030860 con 0x7f30d00b56f0 2024-03-28T17:59:03.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.043+0000 7f30cf400640 1 -- 172.21.15.62:0/3785959077 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034070 con 0x7f30d00b56f0 2024-03-28T17:59:03.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.042+0000 7fbb9f800640 1 Processor -- start 2024-03-28T17:59:03.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.043+0000 7fbb9f800640 1 -- start start 2024-03-28T17:59:03.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 0x7f8d240ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8d2415e260 0x7f8d240d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8d240d37b0 0x7f8d240cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d240d2490 con 0x7f8d2415e260 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d240cf4f0 con 0x7f8d240d37b0 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.044+0000 7f8d29a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240d2710 con 0x7f8d240cebd0 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 0x7f8d240ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f30d6000640 1 -- 172.21.15.62:0/3785959077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 msgr2=0x7f30d00b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f30d6000640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00b5b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009870 tx=0x7f30bc02b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7fbb9f800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb9815b3c0 0x7fbb980aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7fbb9f800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 0x7fbb980af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 0x7f8d240ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54388/0 (socket says 172.21.15.62:54388) 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f8d23e00640 1 -- 172.21.15.62:0/3242240011 learned_addr learned my addr 172.21.15.62:0/3242240011 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f8d29000640 1 --2- 172.21.15.62:0/3242240011 >> v2:172.21.15.81:3300/0 conn(0x7f8d240d37b0 0x7f8d240cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.045+0000 7f8d23400640 1 --2- 172.21.15.62:0/3242240011 >> v2:172.21.15.62:3300/0 conn(0x7f8d2415e260 0x7f8d240d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7f8d29000640 1 -- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 msgr2=0x7f8d240ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7f8d29000640 1 --2- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 0x7f8d240ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7f8d29000640 1 -- 172.21.15.62:0/3242240011 >> v2:172.21.15.62:3300/0 conn(0x7f8d2415e260 msgr2=0x7f8d240d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7f8d29000640 1 --2- 172.21.15.62:0/3242240011 >> v2:172.21.15.62:3300/0 conn(0x7f8d2415e260 0x7f8d240d1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7f8d29000640 1 -- 172.21.15.62:0/3242240011 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240d29f0 con 0x7f8d240d37b0 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7fbb9ce00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbb980b4140 0x7fbb980725a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33802/0 (socket says 172.21.15.62:33802) 2024-03-28T17:59:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7fbb9ce00640 1 -- 172.21.15.62:0/3793109040 learned_addr learned my addr 172.21.15.62:0/3793109040 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7fbb9f800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb98072cb0 con 0x7fbb980b4140 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980af780 con 0x7fbb9815b3c0 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.046+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980b3c50 con 0x7fbb980b4520 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb9d800640 1 --2- 172.21.15.62:0/3793109040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 0x7fbb980af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb97e00640 1 --2- 172.21.15.62:0/3793109040 >> v2:172.21.15.81:3300/0 conn(0x7fbb9815b3c0 0x7fbb980aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7f30d6000640 1 -- 172.21.15.62:0/3785959077 shutdown_connections 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7f30d6000640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7f30d6000640 1 --2- 172.21.15.62:0/3785959077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00b5b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7f30d6000640 1 -- 172.21.15.62:0/3785959077 >> 172.21.15.62:0/3785959077 conn(0x7f30d0071ae0 msgr2=0x7f30d0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb97400640 1 -- 172.21.15.62:0/3793109040 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647328085 0 0) 0x7fbb98072cb0 con 0x7fbb980b4140 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb97400640 1 -- 172.21.15.62:0/3793109040 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7c003520 con 0x7fbb980b4140 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb9d800640 1 -- 172.21.15.62:0/3793109040 >> v2:172.21.15.81:3300/0 conn(0x7fbb9815b3c0 msgr2=0x7fbb980aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb9d800640 1 --2- 172.21.15.62:0/3793109040 >> v2:172.21.15.81:3300/0 conn(0x7fbb9815b3c0 0x7fbb980aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb9d800640 1 -- 172.21.15.62:0/3793109040 >> v1:172.21.15.81:6789/0 conn(0x7fbb980b4140 legacy=0x7fbb980725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.047+0000 7fbb9d800640 1 -- 172.21.15.62:0/3793109040 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980b2e90 con 0x7fbb980b4520 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.048+0000 7f30d6000640 1 -- 172.21.15.62:0/3785959077 shutdown_connections 2024-03-28T17:59:03.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.048+0000 7f30d6000640 1 -- 172.21.15.62:0/3785959077 wait complete. 2024-03-28T17:59:03.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.048+0000 7f8d29000640 1 --2- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d240cefb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d24067970 tx=0x7f8d0c02b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efea071e1b7fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.049+0000 7f8d22a00640 1 -- 172.21.15.62:0/3242240011 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c038070 con 0x7f8d240d37b0 2024-03-28T17:59:03.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.049+0000 7f8d22a00640 1 -- 172.21.15.62:0/3242240011 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c031710 con 0x7f8d240d37b0 2024-03-28T17:59:03.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.049+0000 7f8d22a00640 1 -- 172.21.15.62:0/3242240011 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c031ae0 con 0x7f8d240d37b0 2024-03-28T17:59:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.049+0000 7fbb9d800640 1 --2- 172.21.15.62:0/3793109040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 0x7fbb980af240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009d90 tx=0x7fbb9002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce3005222c8bf0b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 -- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 msgr2=0x7f8d240cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 --2- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d240cefb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d24067970 tx=0x7f8d0c02b7b0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7fbb97400640 1 -- 172.21.15.62:0/3793109040 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90039070 con 0x7fbb980b4520 2024-03-28T17:59:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7fbb97400640 1 -- 172.21.15.62:0/3793109040 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9002cc50 con 0x7fbb980b4520 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7fbb97400640 1 -- 172.21.15.62:0/3793109040 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90034440 con 0x7fbb980b4520 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 -- 172.21.15.62:0/3242240011 shutdown_connections 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 --2- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d240cefb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 --2- 172.21.15.62:0/3242240011 >> v2:172.21.15.62:3300/0 conn(0x7f8d2415e260 0x7f8d240d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 --2- 172.21.15.62:0/3242240011 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d240cebd0 0x7f8d240ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 -- 172.21.15.62:0/3242240011 >> 172.21.15.62:0/3242240011 conn(0x7f8d24071cf0 msgr2=0x7f8d24066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.050+0000 7f8d29a00640 1 -- 172.21.15.62:0/3242240011 shutdown_connections 2024-03-28T17:59:03.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.051+0000 7f8d29a00640 1 -- 172.21.15.62:0/3242240011 wait complete. 2024-03-28T17:59:03.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.052+0000 7f30d6000640 1 Processor -- start 2024-03-28T17:59:03.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.052+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 msgr2=0x7fbb980af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.052+0000 7fbb9f800640 1 --2- 172.21.15.62:0/3793109040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 0x7fbb980af240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009d90 tx=0x7fbb9002bea0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.052+0000 7f30d6000640 1 -- start start 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 shutdown_connections 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 --2- 172.21.15.62:0/3793109040 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4520 0x7fbb980af240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 --2- 172.21.15.62:0/3793109040 >> v2:172.21.15.81:3300/0 conn(0x7fbb9815b3c0 0x7fbb980aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 >> 172.21.15.62:0/3793109040 conn(0x7fbb98066fb0 msgr2=0x7fbb980673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 shutdown_connections 2024-03-28T17:59:03.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.053+0000 7fbb9f800640 1 -- 172.21.15.62:0/3793109040 wait complete. 2024-03-28T17:59:03.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.054+0000 7f8d29a00640 1 Processor -- start 2024-03-28T17:59:03.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.054+0000 7f8d29a00640 1 -- start start 2024-03-28T17:59:03.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.054+0000 7f30d6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.055+0000 7f30d6000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.055+0000 7f30d6000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30d015dd60 0x7f30d015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.055+0000 7f30d6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b79a0 con 0x7f30d00b56f0 2024-03-28T17:59:03.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.055+0000 7f30d6000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b76c0 con 0x7f30d015dd60 2024-03-28T17:59:03.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.055+0000 7f30d6000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b9bc0 con 0x7f30d00b7ec0 2024-03-28T17:59:03.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7f8d29a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7fbb9f800640 1 Processor -- start 2024-03-28T17:59:03.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7f8d29a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d2415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f8d29a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d2415e260 0x7f8d240b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7fbb9f800640 1 -- start start 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47162/0 (socket says 172.21.15.62:47162) 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f8d23400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d2415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7f30cfe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.056+0000 7f30cfe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54404/0 (socket says 172.21.15.62:54404) 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f30cfe00640 1 -- 172.21.15.62:0/2467357860 learned_addr learned my addr 172.21.15.62:0/2467357860 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f30d5600640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30d015dd60 0x7f30d015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f8d23400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d2415fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51610/0 (socket says 172.21.15.62:51610) 2024-03-28T17:59:03.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f8d23e00640 1 -- 172.21.15.62:0/1265361480 learned_addr learned my addr 172.21.15.62:0/1265361480 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.057+0000 7f30d4c00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f8d29000640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d2415e260 0x7f8d240b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f30cfe00640 1 -- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30d015dd60 msgr2=0x7f30d015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f30cfe00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30d015dd60 0x7f30d015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f30cfe00640 1 -- 172.21.15.62:0/2467357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 msgr2=0x7f30d00cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 0x7fbb98152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 0x7fbb981f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9815b3c0 0x7fbb981f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980b3730 con 0x7fbb980b4520 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980b3450 con 0x7fbb9815b3c0 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7fbb9f800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9806ecc0 con 0x7fbb980b4140 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7fbb9ce00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 0x7fbb98152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7fbb9ce00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 0x7fbb98152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54422/0 (socket says 172.21.15.62:54422) 2024-03-28T17:59:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7fbb9ce00640 1 -- 172.21.15.62:0/2197471078 learned_addr learned my addr 172.21.15.62:0/2197471078 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7fbb97e00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 0x7fbb981f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7fbb9d800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9815b3c0 0x7fbb981f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 -- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 msgr2=0x7fbb98152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 0x7fbb98152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 -- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9815b3c0 msgr2=0x7fbb981f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9815b3c0 0x7fbb981f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 -- 172.21.15.62:0/2197471078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98149e80 con 0x7fbb980b4520 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.058+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240d3290 con 0x7f8d240cebd0 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240d2fb0 con 0x7f8d240d37b0 2024-03-28T17:59:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.059+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240b5710 con 0x7f8d2415e260 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7f8d23e00640 1 -- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d2415e260 msgr2=0x7f8d240b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7f8d23e00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d2415e260 0x7f8d240b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7f8d23e00640 1 -- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 msgr2=0x7f8d2415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7f8d23e00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d2415fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.060+0000 7f8d23e00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240c4ad0 con 0x7f8d240cebd0 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7f8d23e00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007bd0 tx=0x7f8d1c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.061+0000 7fbb97e00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 0x7fbb981f5310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e60 tx=0x7fbb8400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.062+0000 7f30cfe00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.062+0000 7f30cfe00640 1 -- 172.21.15.62:0/2467357860 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d00c99b0 con 0x7f30d00b7ec0 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.062+0000 7f30d4c00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f30cfe00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000c00 tx=0x7f30bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c01a070 con 0x7f8d240cebd0 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00dd50 con 0x7f8d240cebd0 2024-03-28T17:59:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c015600 con 0x7f8d240cebd0 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84020070 con 0x7fbb980b4520 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400ee60 con 0x7fbb980b4520 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84020070 con 0x7fbb980b4520 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9814a160 con 0x7fbb980b4520 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9809d470 con 0x7fbb980b4520 2024-03-28T17:59:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240c4db0 con 0x7f8d240cebd0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.063+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d240b6fd0 con 0x7f8d240cebd0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.065+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034040 con 0x7f30d00b7ec0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.065+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc003c50 con 0x7f30d00b7ec0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.065+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc039450 con 0x7f30d00b7ec0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.066+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00c9c90 con 0x7f30d00b7ec0 2024-03-28T17:59:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.067+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d00ca250 con 0x7f30d00b7ec0 2024-03-28T17:59:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.069+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d0072b90 con 0x7f30d00b7ec0 2024-03-28T17:59:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.067+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8d1c01d070 con 0x7f8d240cebd0 2024-03-28T17:59:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.069+0000 7f8d1be00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 0x7f8d0006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.069+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2404f470 con 0x7f8d240cebd0 2024-03-28T17:59:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.069+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d1c08a9f0 con 0x7f8d240cebd0 2024-03-28T17:59:03.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.070+0000 7f8d23400640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 0x7f8d0006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.070+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70005290 con 0x7fbb980b4520 2024-03-28T17:59:03.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.071+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f30bc04b060 con 0x7f30d00b7ec0 2024-03-28T17:59:03.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.073+0000 7f30ccc00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 0x7f30b006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.073+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30bc079270 con 0x7f30d00b7ec0 2024-03-28T17:59:03.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.071+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbb84023070 con 0x7fbb980b4520 2024-03-28T17:59:03.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.074+0000 7fbb94c00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 0x7fbb7806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.074+0000 7f30d4c00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 0x7f30b006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.074+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb8408e2c0 con 0x7fbb980b4520 2024-03-28T17:59:03.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.074+0000 7fbb9ce00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 0x7fbb7806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.075+0000 7f8d23400640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 0x7f8d0006e0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d24150780 tx=0x7f8d14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.075+0000 7f30d4c00640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 0x7f30b006e0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30c400a830 tx=0x7f30c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.076+0000 7fbb9ce00640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 0x7fbb7806e290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005880 tx=0x7fbb88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.079+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbb8405c6c0 con 0x7fbb980b4520 2024-03-28T17:59:03.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.086+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8d1c010070 con 0x7f8d240cebd0 2024-03-28T17:59:03.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.086+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f30bc03f070 con 0x7f30d00b7ec0 2024-03-28T17:59:03.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.116+0000 7fd033200640 1 Processor -- start 2024-03-28T17:59:03.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.116+0000 7fd033200640 1 -- start start 2024-03-28T17:59:03.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.117+0000 7fddd8e00640 1 Processor -- start 2024-03-28T17:59:03.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.117+0000 7fddd8e00640 1 -- start start 2024-03-28T17:59:03.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.118+0000 7fd033200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0280e3040 0x7fd02803c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.118+0000 7fd033200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd028058980 con 0x7fd0280e3420 2024-03-28T17:59:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.118+0000 7fd033200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd028058c60 con 0x7fd0280e78d0 2024-03-28T17:59:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.118+0000 7fd033200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd028006860 con 0x7fd0280e3040 2024-03-28T17:59:03.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.119+0000 7fd031e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0280e3040 0x7fd02803c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.119+0000 7fd031e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0280e3040 0x7fd02803c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47178/0 (socket says 172.21.15.62:47178) 2024-03-28T17:59:03.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.119+0000 7fd031400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd0280e3420 0x7fd028058130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56646/0 (socket says 172.21.15.62:56646) 2024-03-28T17:59:03.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.120+0000 7fd031400640 1 -- 172.21.15.62:0/246079453 learned_addr learned my addr 172.21.15.62:0/246079453 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:03.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.120+0000 7fd030a00640 1 -- 172.21.15.62:0/246079453 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845686802 0 0) 0x7fd028058c60 con 0x7fd0280e78d0 2024-03-28T17:59:03.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fd030a00640 1 -- 172.21.15.62:0/246079453 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd014003520 con 0x7fd0280e78d0 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fddd8e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd40ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fddd8e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fddd8e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd40d24a0 con 0x7fddd4071d10 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fddd8e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40d3330 con 0x7fddd40b3310 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fddd8e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40d2720 con 0x7fddd4071930 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fd031e00640 1 -- 172.21.15.62:0/246079453 >> v1:172.21.15.81:6789/0 conn(0x7fd0280e78d0 legacy=0x7fd028053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fd031e00640 1 -- 172.21.15.62:0/246079453 >> v1:172.21.15.62:6789/0 conn(0x7fd0280e3420 legacy=0x7fd028058130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.121+0000 7fd031e00640 1 -- 172.21.15.62:0/246079453 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028006210 con 0x7fd0280e3040 2024-03-28T17:59:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.122+0000 7fddd1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.122+0000 7fddd1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47190/0 (socket says 172.21.15.62:47190) 2024-03-28T17:59:03.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.122+0000 7fddd1400640 1 -- 172.21.15.62:0/986761862 learned_addr learned my addr 172.21.15.62:0/986761862 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.123+0000 7fddd1e00640 1 --2- 172.21.15.62:0/986761862 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd40ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.124+0000 7fd031e00640 1 --2- 172.21.15.62:0/246079453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3040 0x7fd02803c180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0097a0 tx=0x7fd01c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3771c4273e43a617 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.124+0000 7fddd1e00640 1 -- 172.21.15.62:0/986761862 >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 msgr2=0x7fddd40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.124+0000 7fddd1e00640 1 --2- 172.21.15.62:0/986761862 >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.124+0000 7fddd1e00640 1 -- 172.21.15.62:0/986761862 >> v1:172.21.15.62:6789/0 conn(0x7fddd4071d10 legacy=0x7fddd4162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.124+0000 7fddd1e00640 1 -- 172.21.15.62:0/986761862 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd40d2a00 con 0x7fddd4071930 2024-03-28T17:59:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.125+0000 7fd030a00640 1 -- 172.21.15.62:0/246079453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c035020 con 0x7fd0280e3040 2024-03-28T17:59:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.125+0000 7fddd1400640 1 --2- 172.21.15.62:0/986761862 >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.125+0000 7fd030a00640 1 -- 172.21.15.62:0/246079453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c031870 con 0x7fd0280e3040 2024-03-28T17:59:03.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.125+0000 7fd030a00640 1 -- 172.21.15.62:0/246079453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c035020 con 0x7fd0280e3040 2024-03-28T17:59:03.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.126+0000 7fd033200640 1 -- 172.21.15.62:0/246079453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3040 msgr2=0x7fd02803c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.126+0000 7fd033200640 1 --2- 172.21.15.62:0/246079453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3040 0x7fd02803c180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0097a0 tx=0x7fd01c02b910 comp rx=0 tx=0).stop 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fd033200640 1 -- 172.21.15.62:0/246079453 shutdown_connections 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fd033200640 1 --2- 172.21.15.62:0/246079453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3040 0x7fd02803c180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fd033200640 1 -- 172.21.15.62:0/246079453 >> 172.21.15.62:0/246079453 conn(0x7fd02803aa40 msgr2=0x7fd02803ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fd033200640 1 -- 172.21.15.62:0/246079453 shutdown_connections 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fd033200640 1 -- 172.21.15.62:0/246079453 wait complete. 2024-03-28T17:59:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.127+0000 7fddd1e00640 1 --2- 172.21.15.62:0/986761862 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd40ce7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fddc40097a0 tx=0x7fddc402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca28db91586eab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.128+0000 7fddd0a00640 1 -- 172.21.15.62:0/986761862 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4034070 con 0x7fddd4071930 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.128+0000 7fddd0a00640 1 -- 172.21.15.62:0/986761862 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc4030800 con 0x7fddd4071930 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.128+0000 7fddd0a00640 1 -- 172.21.15.62:0/986761862 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4030bd0 con 0x7fddd4071930 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 -- 172.21.15.62:0/986761862 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 msgr2=0x7fddd40ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 --2- 172.21.15.62:0/986761862 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd40ce7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fddc40097a0 tx=0x7fddc402b940 comp rx=0 tx=0).stop 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 -- 172.21.15.62:0/986761862 shutdown_connections 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 --2- 172.21.15.62:0/986761862 >> v2:172.21.15.62:3300/0 conn(0x7fddd40b3310 0x7fddd40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 --2- 172.21.15.62:0/986761862 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd40ce7e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.129+0000 7fddd8e00640 1 -- 172.21.15.62:0/986761862 >> 172.21.15.62:0/986761862 conn(0x7fddd40d3ef0 msgr2=0x7fddd40d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.130+0000 7fd033200640 1 Processor -- start 2024-03-28T17:59:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.130+0000 7fddd8e00640 1 -- 172.21.15.62:0/986761862 shutdown_connections 2024-03-28T17:59:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.130+0000 7fd033200640 1 -- start start 2024-03-28T17:59:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.132+0000 7fd033200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 0x7fd0280521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.132+0000 7fd033200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0280e78d0 0x7fd028052730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.132+0000 7fd033200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 0x7fd02804f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.130+0000 7fddd8e00640 1 -- 172.21.15.62:0/986761862 wait complete. 2024-03-28T17:59:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fd031e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 0x7fd0280521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fd031e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 0x7fd0280521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47196/0 (socket says 172.21.15.62:47196) 2024-03-28T17:59:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fd031e00640 1 -- 172.21.15.62:0/3990952108 learned_addr learned my addr 172.21.15.62:0/3990952108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fd031400640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0280e78d0 0x7fd028052730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fddd8e00640 1 Processor -- start 2024-03-28T17:59:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.133+0000 7fd032800640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 0x7fd02804f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.134+0000 7fddd8e00640 1 -- start start 2024-03-28T17:59:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.134+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd028054620 con 0x7fd0280e3420 2024-03-28T17:59:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.134+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280544a0 con 0x7fd0280e78d0 2024-03-28T17:59:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.134+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280547a0 con 0x7fd02804eed0 2024-03-28T17:59:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fddd8e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fddd8e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddd4071d10 0x7fddd40aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fddd8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 0x7fddd40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fd032800640 1 -- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0280e78d0 msgr2=0x7fd028052730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fd032800640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0280e78d0 0x7fd028052730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd1e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd1e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:54446/0 (socket says 172.21.15.62:54446) 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd1e00640 1 -- 172.21.15.62:0/1907511598 learned_addr learned my addr 172.21.15.62:0/1907511598 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd1400640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddd4071d10 0x7fddd40aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fd032800640 1 -- 172.21.15.62:0/3990952108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 msgr2=0x7fd0280521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fd032800640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 0x7fd0280521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40b81a0 con 0x7fddd40b3310 2024-03-28T17:59:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40d2fc0 con 0x7fddd4071d10 2024-03-28T17:59:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.137+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40d4dd0 con 0x7fddd4071930 2024-03-28T17:59:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.136+0000 7fd032800640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02804f8a0 con 0x7fd02804eed0 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.138+0000 7fddd2800640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 0x7fddd40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd1e00640 1 -- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddd4071d10 msgr2=0x7fddd40aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd1e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddd4071d10 0x7fddd40aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd1e00640 1 -- 172.21.15.62:0/1907511598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 msgr2=0x7fddd40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd1e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 0x7fddd40b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd1e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd406e5a0 con 0x7fddd4071930 2024-03-28T17:59:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fd032800640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 0x7fd02804f360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd02c050d00 tx=0x7fd02c068ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.139+0000 7fddd2800640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 0x7fddd40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0734a0 con 0x7fd02804eed0 2024-03-28T17:59:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c06f070 con 0x7fd02804eed0 2024-03-28T17:59:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fddd1e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009890 tx=0x7fddc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0943d0 con 0x7fd02804eed0 2024-03-28T17:59:03.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd028186710 con 0x7fd02804eed0 2024-03-28T17:59:03.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd028186cd0 con 0x7fd02804eed0 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.140+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4034040 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.141+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc403add0 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.141+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4006e10 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.141+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd406e2c0 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.141+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd406ff90 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.142+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddac005290 con 0x7fddd4071930 2024-03-28T17:59:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.143+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd008005290 con 0x7fd02804eed0 2024-03-28T17:59:03.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.146+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fddc404b060 con 0x7fddd4071930 2024-03-28T17:59:03.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.148+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd02c094590 con 0x7fd02804eed0 2024-03-28T17:59:03.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.148+0000 7fddca000640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 0x7fddb406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.148+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddc40acd30 con 0x7fddd4071930 2024-03-28T17:59:03.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.148+0000 7fddd1400640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 0x7fddb406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.149+0000 7fd026000640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 0x7fd01006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.150+0000 7fddd1400640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 0x7fddb406e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cb5c0 tx=0x7fddc0009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.149+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd02c105c50 con 0x7fd02804eed0 2024-03-28T17:59:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.150+0000 7fd031e00640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 0x7fd01006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.151+0000 7fd031e00640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 0x7fd01006e160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0097a0 tx=0x7fd01c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.162+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd02c071050 con 0x7fd02804eed0 2024-03-28T17:59:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.172+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fddc403f070 con 0x7fddd4071930 2024-03-28T17:59:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.173+0000 7f951e800640 1 Processor -- start 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.173+0000 7f951e800640 1 -- start start 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.175+0000 7f951e800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f95180b8a80 0x7f95180b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.175+0000 7f951e800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95181618e0 con 0x7f95180b9c60 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.175+0000 7f951e800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95180b79e0 con 0x7f9518161500 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.175+0000 7f951e800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95180b85f0 con 0x7f95180b8a80 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.176+0000 7f9517400640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9518161500 0x7f95180d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:33826/0 (socket says 172.21.15.62:33826) 2024-03-28T17:59:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.176+0000 7f9517400640 1 -- 172.21.15.62:0/769004960 learned_addr learned my addr 172.21.15.62:0/769004960 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.176+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546570268 0 0) 0x7f95180b79e0 con 0x7f9518161500 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.176+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94fc003520 con 0x7f9518161500 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.177+0000 7f9517e00640 1 --2- 172.21.15.62:0/769004960 >> v2:172.21.15.81:3300/0 conn(0x7f95180b8a80 0x7f95180b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.177+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2416665394 0 0) 0x7f94fc003520 con 0x7f9518161500 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.177+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 >> v2:172.21.15.81:3300/0 conn(0x7f95180b8a80 msgr2=0x7f95180b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.177+0000 7f9516000640 1 --2- 172.21.15.62:0/769004960 >> v2:172.21.15.81:3300/0 conn(0x7f95180b8a80 0x7f95180b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.177+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 >> v1:172.21.15.62:6789/0 conn(0x7f95180b9c60 legacy=0x7f95180b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180b7cc0 con 0x7f9518161500 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f9504001650 con 0x7f9518161500 2024-03-28T17:59:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 >> v1:172.21.15.81:6789/0 conn(0x7f9518161500 legacy=0x7f95180d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 0x7f94fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc0052a0 0x7f94fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180b85f0 con 0x7f94fc0052a0 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00c1d0 con 0x7f94fc004800 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.178+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00c3d0 con 0x7f94fc003a00 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 0x7f94fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9516a00640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517e00640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc0052a0 0x7f94fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 -- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94fc004800 msgr2=0x7f94fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 -- 172.21.15.62:0/769004960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc0052a0 msgr2=0x7f94fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc0052a0 0x7f94fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517400640 1 -- 172.21.15.62:0/769004960 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc003520 con 0x7f94fc003a00 2024-03-28T17:59:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.179+0000 7f9517e00640 1 -- 172.21.15.62:0/769004960 reap_dead start 2024-03-28T17:59:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.181+0000 7f9517400640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 0x7f94fc003e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9504007f90 tx=0x7f94fc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec176de0bc0368b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.182+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504007be0 con 0x7f94fc003a00 2024-03-28T17:59:03.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.182+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9504030030 con 0x7f94fc003a00 2024-03-28T17:59:03.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.182+0000 7f9516000640 1 -- 172.21.15.62:0/769004960 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504004d20 con 0x7f94fc003a00 2024-03-28T17:59:03.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.183+0000 7f951e800640 1 -- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 msgr2=0x7f94fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.183+0000 7f951e800640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 0x7f94fc003e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9504007f90 tx=0x7f94fc005830 comp rx=0 tx=0).stop 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.183+0000 7f951e800640 1 -- 172.21.15.62:0/769004960 shutdown_connections 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.183+0000 7f951e800640 1 --2- 172.21.15.62:0/769004960 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94fc003a00 0x7f94fc003e20 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9504007f90 tx=0x7f94fc005830 comp rx=0 tx=0).stop 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.184+0000 7f5cd2a00640 1 Processor -- start 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.183+0000 7f951e800640 1 -- 172.21.15.62:0/769004960 >> 172.21.15.62:0/769004960 conn(0x7f9518071ae0 msgr2=0x7f9518071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.184+0000 7f5cd2a00640 1 -- start start 2024-03-28T17:59:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.184+0000 7f951e800640 1 -- 172.21.15.62:0/769004960 shutdown_connections 2024-03-28T17:59:03.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.184+0000 7f951e800640 1 -- 172.21.15.62:0/769004960 wait complete. 2024-03-28T17:59:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.186+0000 7f5cd2a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ccc06e030 0x7f5ccc0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.186+0000 7f5cd2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5ccc162550 0x7f5ccc0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5cd2a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ccc15e660 con 0x7f5ccc06e410 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5cd2a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc1629e0 con 0x7f5ccc162550 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5cd2a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc071470 con 0x7f5ccc06e030 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5ccbe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ccc06e030 0x7f5ccc0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5ccbe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ccc06e030 0x7f5ccc0aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51662/0 (socket says 172.21.15.62:51662) 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2480051177 learned_addr learned my addr 172.21.15.62:0/2480051177 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5ccb400640 1 --2- 172.21.15.62:0/2480051177 >> v2:172.21.15.62:3300/0 conn(0x7f5ccc162550 0x7f5ccc0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.187+0000 7f5ccaa00640 1 -- 172.21.15.62:0/2480051177 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827031683 0 0) 0x7f5ccc15e660 con 0x7f5ccc06e410 2024-03-28T17:59:03.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f951e800640 1 Processor -- start 2024-03-28T17:59:03.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f5ccaa00640 1 -- 172.21.15.62:0/2480051177 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb0003520 con 0x7f5ccc06e410 2024-03-28T17:59:03.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2480051177 >> v2:172.21.15.62:3300/0 conn(0x7f5ccc162550 msgr2=0x7f5ccc0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f5ccbe00640 1 --2- 172.21.15.62:0/2480051177 >> v2:172.21.15.62:3300/0 conn(0x7f5ccc162550 0x7f5ccc0ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2480051177 >> v1:172.21.15.81:6789/0 conn(0x7f5ccc06e410 legacy=0x7f5ccc0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2480051177 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc071750 con 0x7f5ccc06e030 2024-03-28T17:59:03.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.188+0000 7f951e800640 1 -- start start 2024-03-28T17:59:03.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.190+0000 7f5ccbe00640 1 --2- 172.21.15.62:0/2480051177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc0aed10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009870 tx=0x7f5cb802b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafaf93a3a186b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.190+0000 7f951e800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9518147eb0 0x7f95180cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.190+0000 7f5ccaa00640 1 -- 172.21.15.62:0/2480051177 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8034070 con 0x7f5ccc06e030 2024-03-28T17:59:03.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.190+0000 7f5ccaa00640 1 -- 172.21.15.62:0/2480051177 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8030870 con 0x7f5ccc06e030 2024-03-28T17:59:03.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.190+0000 7f951e800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95181f4300 0x7f95181f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f951e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f5ccaa00640 1 -- 172.21.15.62:0/2480051177 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8030c80 con 0x7f5ccc06e030 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f9517400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9518147eb0 0x7f95180cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f951e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180b8ed0 con 0x7f951815bb30 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f951e800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180b8280 con 0x7f9518147eb0 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f9517e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f9517e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47228/0 (socket says 172.21.15.62:47228) 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f9517e00640 1 -- 172.21.15.62:0/2274545962 learned_addr learned my addr 172.21.15.62:0/2274545962 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.191+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95180b9050 con 0x7f95181f4300 2024-03-28T17:59:03.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9516a00640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95181f4300 0x7f95181f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2480051177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 msgr2=0x7f5ccc0aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2480051177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc0aed10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009870 tx=0x7f5cb802b9b0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2480051177 shutdown_connections 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2480051177 >> v2:172.21.15.62:3300/0 conn(0x7f5ccc162550 0x7f5ccc0ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2480051177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc0aed10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2480051177 >> 172.21.15.62:0/2480051177 conn(0x7f5ccc066dd0 msgr2=0x7f5ccc0671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.193+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2480051177 shutdown_connections 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9517e00640 1 -- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95181f4300 msgr2=0x7f95181f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9517e00640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95181f4300 0x7f95181f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9517e00640 1 -- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9518147eb0 msgr2=0x7f95180cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9517e00640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9518147eb0 0x7f95180cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.192+0000 7f9517e00640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951815d520 con 0x7f951815bb30 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.193+0000 7f9517e00640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f95180b6870 tx=0x7f9510009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.193+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2480051177 wait complete. 2024-03-28T17:59:03.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.195+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510018de0 con 0x7f951815bb30 2024-03-28T17:59:03.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.195+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951001c070 con 0x7f951815bb30 2024-03-28T17:59:03.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.195+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510007b50 con 0x7f951815bb30 2024-03-28T17:59:03.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.196+0000 7f5cd2a00640 1 Processor -- start 2024-03-28T17:59:03.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.197+0000 7f5cd2a00640 1 -- start start 2024-03-28T17:59:03.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.199+0000 7f5cd2a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.199+0000 7f5cd2a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ccc06e410 0x7f5ccc2010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.195+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951815d800 con 0x7f951815bb30 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.196+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951815ddc0 con 0x7f951815bb30 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f5cd2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 0x7f5ccc1fe220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f5ccbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f5ccbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51684/0 (socket says 172.21.15.62:51684) 2024-03-28T17:59:03.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2937798706 learned_addr learned my addr 172.21.15.62:0/2937798706 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f5cd0a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 0x7f5ccc1fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.200+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9510007cf0 con 0x7f951815bb30 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.201+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0b40e0 con 0x7f5ccc154c40 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.201+0000 7f5ccb400640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ccc06e410 0x7f5ccc2010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.201+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc071a30 con 0x7f5ccc06e030 2024-03-28T17:59:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.201+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0b4260 con 0x7f5ccc06e410 2024-03-28T17:59:03.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ccc06e410 msgr2=0x7f5ccc2010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f5ccbe00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ccc06e410 0x7f5ccc2010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2937798706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 msgr2=0x7f5ccc1fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f5ccbe00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 0x7f5ccc1fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.204+0000 7f5ccbe00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc201e30 con 0x7f5ccc06e030 2024-03-28T17:59:03.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.204+0000 7f5cd0a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 0x7f5ccc1fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:03.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.204+0000 7f5ccbe00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.201+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95180d4780 con 0x7f951815bb30 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f950f400640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 0x7f94f806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.203+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95100933d0 con 0x7f951815bb30 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.204+0000 7f9517400640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 0x7f94f806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.205+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8048070 con 0x7f5ccc06e030 2024-03-28T17:59:03.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.205+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc202110 con 0x7f5ccc06e030 2024-03-28T17:59:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.205+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc2026d0 con 0x7f5ccc06e030 2024-03-28T17:59:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.205+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8030670 con 0x7f5ccc06e030 2024-03-28T17:59:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.205+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8034040 con 0x7f5ccc06e030 2024-03-28T17:59:03.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.207+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca4005290 con 0x7f5ccc06e030 2024-03-28T17:59:03.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.209+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5cb804b070 con 0x7f5ccc06e030 2024-03-28T17:59:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.211+0000 7f5cc3e00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 0x7f5cac06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.211+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cb80acc40 con 0x7f5ccc06e030 2024-03-28T17:59:03.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.213+0000 7f5ccb400640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 0x7f5cac06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.212+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f95180d4780 con 0x7f951815bb30 2024-03-28T17:59:03.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.213+0000 7f9517400640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 0x7f94f806e0e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9504034000 tx=0x7f9504009070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.215+0000 7f5ccb400640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 0x7f5cac06e090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.223+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5cb8052080 con 0x7f5ccc06e030 2024-03-28T17:59:03.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.474+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd80161a30 con 0x7fbd800b3310 2024-03-28T17:59:03.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.475+0000 7fbd77400640 1 -- 172.21.15.62:0/226362894 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd6c058e00 con 0x7fbd800b3310 2024-03-28T17:59:03.476 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:03.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.478+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 msgr2=0x7fbd6006e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.478+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 0x7fbd6006e190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd800d3ec0 tx=0x7fbd78007d50 comp rx=0 tx=0).stop 2024-03-28T17:59:03.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 msgr2=0x7fbd800b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e850 tx=0x7fbd6c00ed10 comp rx=0 tx=0).stop 2024-03-28T17:59:03.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 shutdown_connections 2024-03-28T17:59:03.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbd6006bcd0 0x7fbd6006e190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd800b3310 0x7fbd800b5e70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd80071d10 0x7fbd800708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 --2- 172.21.15.62:0/226362894 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbd80071930 0x7fbd801614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.479+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 >> 172.21.15.62:0/226362894 conn(0x7fbd800d3ef0 msgr2=0x7fbd800cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.480+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 shutdown_connections 2024-03-28T17:59:03.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.480+0000 7fbd86600640 1 -- 172.21.15.62:0/226362894 wait complete. 2024-03-28T17:59:03.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.482+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb70005580 con 0x7fbb980b4520 2024-03-28T17:59:03.487 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.483+0000 7fbb94c00640 1 -- 172.21.15.62:0/2197471078 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb8408e0c0 con 0x7fbb980b4520 2024-03-28T17:59:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.486+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 msgr2=0x7fbb7806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.486+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 0x7fbb7806e290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005880 tx=0x7fbb88009040 comp rx=0 tx=0).stop 2024-03-28T17:59:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.486+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 msgr2=0x7fbb981f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.486+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 0x7fbb981f5310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e60 tx=0x7fbb8400ec70 comp rx=0 tx=0).stop 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 shutdown_connections 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb7806bdd0 0x7fbb7806e290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9815b3c0 0x7fbb981f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980b4520 0x7fbb981f5310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 --2- 172.21.15.62:0/2197471078 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb980b4140 0x7fbb98152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.487+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 >> 172.21.15.62:0/2197471078 conn(0x7fbb98066fb0 msgr2=0x7fbb98067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.488+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 shutdown_connections 2024-03-28T17:59:03.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.489+0000 7fbb9f800640 1 -- 172.21.15.62:0/2197471078 wait complete. 2024-03-28T17:59:03.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-03-28T17:59:03.510 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.510+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f30d004f4e0 con 0x7f30d00b7ec0 2024-03-28T17:59:03.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.511+0000 7f30ccc00640 1 -- 172.21.15.62:0/2467357860 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30bc07b8d0 con 0x7f30d00b7ec0 2024-03-28T17:59:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-03-28T17:59:03.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.515+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 msgr2=0x7f30b006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.515+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 0x7f30b006e0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30c400a830 tx=0x7f30c4008040 comp rx=0 tx=0).stop 2024-03-28T17:59:03.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.515+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 msgr2=0x7f30d00c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.515+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000c00 tx=0x7f30bc0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-28T17:59:03.517 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 shutdown_connections 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f30b006bc20 0x7f30b006e0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30d015dd60 0x7f30d015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f30d00b7ec0 0x7f30d00c9080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 --2- 172.21.15.62:0/2467357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d00b56f0 0x7f30d00cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 >> 172.21.15.62:0/2467357860 conn(0x7f30d0071ae0 msgr2=0x7f30d00d4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 shutdown_connections 2024-03-28T17:59:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.516+0000 7f30d6000640 1 -- 172.21.15.62:0/2467357860 wait complete. 2024-03-28T17:59:03.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.520+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8d24068820 con 0x7f8d240cebd0 2024-03-28T17:59:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.521+0000 7f8d1be00640 1 -- 172.21.15.62:0/1265361480 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d1c05c020 con 0x7f8d240cebd0 2024-03-28T17:59:03.522 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:03.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.523+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 msgr2=0x7f8d0006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.523+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 0x7f8d0006e0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d24150780 tx=0x7f8d14008040 comp rx=0 tx=0).stop 2024-03-28T17:59:03.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.523+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 msgr2=0x7f8d240c0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.523+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007bd0 tx=0x7f8d1c00d4a0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 shutdown_connections 2024-03-28T17:59:03.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8d0006bc20 0x7f8d0006e0e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8d2415e260 0x7f8d240b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d240d37b0 0x7f8d2415fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 --2- 172.21.15.62:0/1265361480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d240cebd0 0x7f8d240c0f40 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007bd0 tx=0x7f8d1c00d4a0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 >> 172.21.15.62:0/1265361480 conn(0x7f8d24071cf0 msgr2=0x7f8d240b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 shutdown_connections 2024-03-28T17:59:03.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.524+0000 7f8d29a00640 1 -- 172.21.15.62:0/1265361480 wait complete. 2024-03-28T17:59:03.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-03-28T17:59:03.541 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-03-28T17:59:03.542 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.578+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5ca4005850 con 0x7f5ccc06e030 2024-03-28T17:59:03.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.579+0000 7f5cc3e00640 1 -- 172.21.15.62:0/2937798706 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cb807b150 con 0x7f5ccc06e030 2024-03-28T17:59:03.580 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 msgr2=0x7f5cac06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 0x7f5cac06e090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc008040 comp rx=0 tx=0).stop 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 msgr2=0x7f5ccc2017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8004730 comp rx=0 tx=0).stop 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 shutdown_connections 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cac06bbd0 0x7f5cac06e090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ccc154c40 0x7f5ccc1fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ccc06e410 0x7f5ccc2010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 --2- 172.21.15.62:0/2937798706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ccc06e030 0x7f5ccc2017a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 >> 172.21.15.62:0/2937798706 conn(0x7f5ccc066dd0 msgr2=0x7f5ccc0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 shutdown_connections 2024-03-28T17:59:03.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.581+0000 7f5cd2a00640 1 -- 172.21.15.62:0/2937798706 wait complete. 2024-03-28T17:59:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.586+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f95180680c0 con 0x7f951815bb30 2024-03-28T17:59:03.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.587+0000 7f950f400640 1 -- 172.21.15.62:0/2274545962 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9510061a00 con 0x7f951815bb30 2024-03-28T17:59:03.588 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 msgr2=0x7f94f806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 0x7f94f806e0e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9504034000 tx=0x7f9504009070 comp rx=0 tx=0).stop 2024-03-28T17:59:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 msgr2=0x7f951815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f95180b6870 tx=0x7f9510009d20 comp rx=0 tx=0).stop 2024-03-28T17:59:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 shutdown_connections 2024-03-28T17:59:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94f806bc20 0x7f94f806e0e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f951815bb30 0x7f951815bfb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95181f4300 0x7f95181f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 --2- 172.21.15.62:0/2274545962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9518147eb0 0x7f95180cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 >> 172.21.15.62:0/2274545962 conn(0x7f9518071ae0 msgr2=0x7f9518161af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 shutdown_connections 2024-03-28T17:59:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.589+0000 7f951e800640 1 -- 172.21.15.62:0/2274545962 wait complete. 2024-03-28T17:59:03.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-03-28T17:59:03.599 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.600+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd008005850 con 0x7fd02804eed0 2024-03-28T17:59:03.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.601+0000 7fd026000640 1 -- 172.21.15.62:0/3990952108 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd02c0d4160 con 0x7fd02804eed0 2024-03-28T17:59:03.602 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 msgr2=0x7fd01006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 0x7fd01006e160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0097a0 tx=0x7fd01c03d040 comp rx=0 tx=0).stop 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 msgr2=0x7fd02804f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 0x7fd02804f360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd02c050d00 tx=0x7fd02c068ba0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 shutdown_connections 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd01006bca0 0x7fd01006e160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd02804eed0 0x7fd02804f360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0280e78d0 0x7fd028052730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 --2- 172.21.15.62:0/3990952108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0280e3420 0x7fd0280521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 >> 172.21.15.62:0/3990952108 conn(0x7fd02803aa40 msgr2=0x7fd028057b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 shutdown_connections 2024-03-28T17:59:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.604+0000 7fd033200640 1 -- 172.21.15.62:0/3990952108 wait complete. 2024-03-28T17:59:03.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-28T17:59:03.607 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.614+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fddac005850 con 0x7fddd4071930 2024-03-28T17:59:03.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.615+0000 7fddca000640 1 -- 172.21.15.62:0/1907511598 <== mon.2 v2:172.21.15.81:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fddc407b360 con 0x7fddd4071930 2024-03-28T17:59:03.616 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 msgr2=0x7fddb406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 0x7fddb406e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cb5c0 tx=0x7fddc0009240 comp rx=0 tx=0).stop 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 msgr2=0x7fddd4161a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009890 tx=0x7fddc40046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 shutdown_connections 2024-03-28T17:59:03.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fddb406bca0 0x7fddb406e160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddd40b3310 0x7fddd40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddd4071d10 0x7fddd40aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 --2- 172.21.15.62:0/1907511598 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fddd4071930 0x7fddd4161a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 >> 172.21.15.62:0/1907511598 conn(0x7fddd40d3ef0 msgr2=0x7fddd40d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 shutdown_connections 2024-03-28T17:59:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.618+0000 7fddd8e00640 1 -- 172.21.15.62:0/1907511598 wait complete. 2024-03-28T17:59:03.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-28T17:59:03.622 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-03-28T17:59:03.637 DEBUG:teuthology.parallel:result is None 2024-03-28T17:59:03.637 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T17:59:03.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T17:59:03.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.887+0000 7eff03e00640 1 Processor -- start 2024-03-28T17:59:03.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.888+0000 7eff03e00640 1 -- start start 2024-03-28T17:59:03.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.889+0000 7eff03e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7eff040b5b50 0x7eff040b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.889+0000 7eff03e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff040b6e60 con 0x7eff0415dd60 2024-03-28T17:59:03.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.889+0000 7eff02a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7eff0415dd60 0x7eff040b94b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53694/0 (socket says 172.21.15.62:53694) 2024-03-28T17:59:03.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff02a00640 1 -- 172.21.15.62:0/3201757081 learned_addr learned my addr 172.21.15.62:0/3201757081 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:03.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff040b7100 con 0x7eff040b56f0 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03400640 1 --2- 172.21.15.62:0/3201757081 >> v2:172.21.15.62:3300/0 conn(0x7eff040b5b50 0x7eff040b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040b6c30 con 0x7eff040b5b50 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03400640 1 -- 172.21.15.62:0/3201757081 >> v1:172.21.15.81:6789/0 conn(0x7eff040b56f0 legacy=0x7eff040b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03400640 1 -- 172.21.15.62:0/3201757081 >> v1:172.21.15.62:6789/0 conn(0x7eff0415dd60 legacy=0x7eff040b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff03400640 1 -- 172.21.15.62:0/3201757081 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040b73e0 con 0x7eff040b5b50 2024-03-28T17:59:03.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.890+0000 7eff02000640 1 -- 172.21.15.62:0/3201757081 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922482612 0 0) 0x7eff040b6e60 con 0x7eff0415dd60 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.891+0000 7eff03400640 1 --2- 172.21.15.62:0/3201757081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff040b85d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d90 tx=0x7efefc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3175b92289c4e181 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.891+0000 7eff02000640 1 -- 172.21.15.62:0/3201757081 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc039070 con 0x7eff040b5b50 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.891+0000 7eff02000640 1 -- 172.21.15.62:0/3201757081 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc02cd00 con 0x7eff040b5b50 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.891+0000 7eff02000640 1 -- 172.21.15.62:0/3201757081 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc034660 con 0x7eff040b5b50 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 msgr2=0x7eff040b85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 --2- 172.21.15.62:0/3201757081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff040b85d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efefc009d90 tx=0x7efefc02bf50 comp rx=0 tx=0).stop 2024-03-28T17:59:03.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 shutdown_connections 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 --2- 172.21.15.62:0/3201757081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff040b85d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 >> 172.21.15.62:0/3201757081 conn(0x7eff04071ae0 msgr2=0x7eff04071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 shutdown_connections 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.892+0000 7eff03e00640 1 -- 172.21.15.62:0/3201757081 wait complete. 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.894+0000 7eff03e00640 1 Processor -- start 2024-03-28T17:59:03.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.894+0000 7eff03e00640 1 -- start start 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 0x7eff0415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 0x7eff040c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff040b9bc0 con 0x7eff040b5b50 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff040b79a0 con 0x7eff040b56f0 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0415e140 con 0x7eff0415dd60 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58162/0 (socket says 172.21.15.62:58162) 2024-03-28T17:59:03.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 -- 172.21.15.62:0/1577093796 learned_addr learned my addr 172.21.15.62:0/1577093796 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff02a00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 0x7eff0415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03400640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 0x7eff040c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 -- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 msgr2=0x7eff040c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 0x7eff040c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 -- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 msgr2=0x7eff0415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 0x7eff0415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7efefa000640 1 -- 172.21.15.62:0/1577093796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040c8c30 con 0x7eff040b5b50 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff02a00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 0x7eff0415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.895+0000 7eff03400640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 0x7eff040c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7efefa000640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c430 tx=0x7efeec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec011790 con 0x7eff040b5b50 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec017070 con 0x7eff040b5b50 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec011bb0 con 0x7eff040b5b50 2024-03-28T17:59:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff040c8f10 con 0x7eff040b5b50 2024-03-28T17:59:03.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.896+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff040c94d0 con 0x7eff040b5b50 2024-03-28T17:59:03.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.897+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff040cfac0 con 0x7eff040b5b50 2024-03-28T17:59:03.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.898+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efeec01c460 con 0x7eff040b5b50 2024-03-28T17:59:03.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.899+0000 7efefb400640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 0x7efee006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:03.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.899+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeec08a960 con 0x7eff040b5b50 2024-03-28T17:59:03.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.899+0000 7eff02a00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 0x7efee006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:03.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.901+0000 7eff02a00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 0x7efee006e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efef4000f60 tx=0x7efef4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:03.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:03.903+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7eff040cfac0 con 0x7eff040b5b50 2024-03-28T17:59:04.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.095+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff040b9970 con 0x7efee006bbd0 2024-03-28T17:59:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.119+0000 7efefb400640 1 -- 172.21.15.62:0/1577093796 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7eff040b9970 con 0x7efee006bbd0 2024-03-28T17:59:04.124 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:04.126 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T17:59:04.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 msgr2=0x7efee006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 0x7efee006e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efef4000f60 tx=0x7efef4009040 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 msgr2=0x7eff0414d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c430 tx=0x7efeec00c8f0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 shutdown_connections 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efee006bbd0 0x7efee006e090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7eff0415dd60 0x7eff040c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff040b5b50 0x7eff0414d2c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 --2- 172.21.15.62:0/1577093796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff040b56f0 0x7eff0415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 >> 172.21.15.62:0/1577093796 conn(0x7eff04071ae0 msgr2=0x7eff040d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:04.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 shutdown_connections 2024-03-28T17:59:04.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.125+0000 7eff03e00640 1 -- 172.21.15.62:0/1577093796 wait complete. 2024-03-28T17:59:04.144 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-28T17:59:02.306930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.333214"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096023+0000","last_change":"2024-03-28T17:58:54.011100+0000","last_active":"2024-03-28T17:58:55.096023+0000","last_peered":"2024-03-28T17:58:55.096023+0000","last_clean":"2024-03-28T17:58:55.096023+0000","last_became_active":"2024-03-28T17:58:54.010769+0000","last_became_peered":"2024-03-28T17:58:54.010769+0000","last_unstale":"2024-03-28T17:58:55.096023+0000","last_undegraded":"2024-03-28T17:58:55.096023+0000","last_fullsized":"2024-03-28T17:58:55.096023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999288+0000","last_change":"2024-03-28T17:58:54.016056+0000","last_active":"2024-03-28T17:58:54.999288+0000","last_peered":"2024-03-28T17:58:54.999288+0000","last_clean":"2024-03-28T17:58:54.999288+0000","last_became_active":"2024-03-28T17:58:54.012351+0000","last_became_peered":"2024-03-28T17:58:54.012351+0000","last_unstale":"2024-03-28T17:58:54.999288+0000","last_undegraded":"2024-03-28T17:58:54.999288+0000","last_fullsized":"2024-03-28T17:58:54.999288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097234+0000","last_change":"2024-03-28T17:58:53.996524+0000","last_active":"2024-03-28T17:58:55.097234+0000","last_peered":"2024-03-28T17:58:55.097234+0000","last_clean":"2024-03-28T17:58:55.097234+0000","last_became_active":"2024-03-28T17:58:53.996197+0000","last_became_peered":"2024-03-28T17:58:53.996197+0000","last_unstale":"2024-03-28T17:58:55.097234+0000","last_undegraded":"2024-03-28T17:58:55.097234+0000","last_fullsized":"2024-03-28T17:58:55.097234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:16.038136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.001103+0000","last_change":"2024-03-28T17:58:54.009958+0000","last_active":"2024-03-28T17:58:55.001103+0000","last_peered":"2024-03-28T17:58:55.001103+0000","last_clean":"2024-03-28T17:58:55.001103+0000","last_became_active":"2024-03-28T17:58:54.009279+0000","last_became_peered":"2024-03-28T17:58:54.009279+0000","last_unstale":"2024-03-28T17:58:55.001103+0000","last_undegraded":"2024-03-28T17:58:55.001103+0000","last_fullsized":"2024-03-28T17:58:55.001103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095127+0000","last_change":"2024-03-28T17:58:54.007553+0000","last_active":"2024-03-28T17:58:55.095127+0000","last_peered":"2024-03-28T17:58:55.095127+0000","last_clean":"2024-03-28T17:58:55.095127+0000","last_became_active":"2024-03-28T17:58:54.007177+0000","last_became_peered":"2024-03-28T17:58:54.007177+0000","last_unstale":"2024-03-28T17:58:55.095127+0000","last_undegraded":"2024-03-28T17:58:55.095127+0000","last_fullsized":"2024-03-28T17:58:55.095127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999258+0000","last_change":"2024-03-28T17:58:53.998917+0000","last_active":"2024-03-28T17:58:54.999258+0000","last_peered":"2024-03-28T17:58:54.999258+0000","last_clean":"2024-03-28T17:58:54.999258+0000","last_became_active":"2024-03-28T17:58:53.998692+0000","last_became_peered":"2024-03-28T17:58:53.998692+0000","last_unstale":"2024-03-28T17:58:54.999258+0000","last_undegraded":"2024-03-28T17:58:54.999258+0000","last_fullsized":"2024-03-28T17:58:54.999258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094845+0000","last_change":"2024-03-28T17:58:54.007407+0000","last_active":"2024-03-28T17:58:55.094845+0000","last_peered":"2024-03-28T17:58:55.094845+0000","last_clean":"2024-03-28T17:58:55.094845+0000","last_became_active":"2024-03-28T17:58:54.007126+0000","last_became_peered":"2024-03-28T17:58:54.007126+0000","last_unstale":"2024-03-28T17:58:55.094845+0000","last_undegraded":"2024-03-28T17:58:55.094845+0000","last_fullsized":"2024-03-28T17:58:55.094845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094632+0000","last_change":"2024-03-28T17:58:54.021995+0000","last_active":"2024-03-28T17:58:55.094632+0000","last_peered":"2024-03-28T17:58:55.094632+0000","last_clean":"2024-03-28T17:58:55.094632+0000","last_became_active":"2024-03-28T17:58:54.021661+0000","last_became_peered":"2024-03-28T17:58:54.021661+0000","last_unstale":"2024-03-28T17:58:55.094632+0000","last_undegraded":"2024-03-28T17:58:55.094632+0000","last_fullsized":"2024-03-28T17:58:55.094632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096407+0000","last_change":"2024-03-28T17:58:54.014331+0000","last_active":"2024-03-28T17:58:55.096407+0000","last_peered":"2024-03-28T17:58:55.096407+0000","last_clean":"2024-03-28T17:58:55.096407+0000","last_became_active":"2024-03-28T17:58:54.014014+0000","last_became_peered":"2024-03-28T17:58:54.014014+0000","last_unstale":"2024-03-28T17:58:55.096407+0000","last_undegraded":"2024-03-28T17:58:55.096407+0000","last_fullsized":"2024-03-28T17:58:55.096407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019925+0000","last_change":"2024-03-28T17:58:54.013693+0000","last_active":"2024-03-28T17:58:55.019925+0000","last_peered":"2024-03-28T17:58:55.019925+0000","last_clean":"2024-03-28T17:58:55.019925+0000","last_became_active":"2024-03-28T17:58:54.013397+0000","last_became_peered":"2024-03-28T17:58:54.013397+0000","last_unstale":"2024-03-28T17:58:55.019925+0000","last_undegraded":"2024-03-28T17:58:55.019925+0000","last_fullsized":"2024-03-28T17:58:55.019925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019799+0000","last_change":"2024-03-28T17:58:54.014516+0000","last_active":"2024-03-28T17:58:55.019799+0000","last_peered":"2024-03-28T17:58:55.019799+0000","last_clean":"2024-03-28T17:58:55.019799+0000","last_became_active":"2024-03-28T17:58:54.014122+0000","last_became_peered":"2024-03-28T17:58:54.014122+0000","last_unstale":"2024-03-28T17:58:55.019799+0000","last_undegraded":"2024-03-28T17:58:55.019799+0000","last_fullsized":"2024-03-28T17:58:55.019799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097339+0000","last_change":"2024-03-28T17:58:54.015180+0000","last_active":"2024-03-28T17:58:55.097339+0000","last_peered":"2024-03-28T17:58:55.097339+0000","last_clean":"2024-03-28T17:58:55.097339+0000","last_became_active":"2024-03-28T17:58:54.008685+0000","last_became_peered":"2024-03-28T17:58:54.008685+0000","last_unstale":"2024-03-28T17:58:55.097339+0000","last_undegraded":"2024-03-28T17:58:55.097339+0000","last_fullsized":"2024-03-28T17:58:55.097339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095295+0000","last_change":"2024-03-28T17:58:54.012021+0000","last_active":"2024-03-28T17:58:55.095295+0000","last_peered":"2024-03-28T17:58:55.095295+0000","last_clean":"2024-03-28T17:58:55.095295+0000","last_became_active":"2024-03-28T17:58:54.011650+0000","last_became_peered":"2024-03-28T17:58:54.011650+0000","last_unstale":"2024-03-28T17:58:55.095295+0000","last_undegraded":"2024-03-28T17:58:55.095295+0000","last_fullsized":"2024-03-28T17:58:55.095295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097361+0000","last_change":"2024-03-28T17:58:54.011097+0000","last_active":"2024-03-28T17:58:55.097361+0000","last_peered":"2024-03-28T17:58:55.097361+0000","last_clean":"2024-03-28T17:58:55.097361+0000","last_became_active":"2024-03-28T17:58:54.010790+0000","last_became_peered":"2024-03-28T17:58:54.010790+0000","last_unstale":"2024-03-28T17:58:55.097361+0000","last_undegraded":"2024-03-28T17:58:55.097361+0000","last_fullsized":"2024-03-28T17:58:55.097361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:51:58.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996815+0000","last_change":"2024-03-28T17:58:54.022714+0000","last_active":"2024-03-28T17:58:54.996815+0000","last_peered":"2024-03-28T17:58:54.996815+0000","last_clean":"2024-03-28T17:58:54.996815+0000","last_became_active":"2024-03-28T17:58:54.022409+0000","last_became_peered":"2024-03-28T17:58:54.022409+0000","last_unstale":"2024-03-28T17:58:54.996815+0000","last_undegraded":"2024-03-28T17:58:54.996815+0000","last_fullsized":"2024-03-28T17:58:54.996815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999073+0000","last_change":"2024-03-28T17:58:54.005916+0000","last_active":"2024-03-28T17:58:54.999073+0000","last_peered":"2024-03-28T17:58:54.999073+0000","last_clean":"2024-03-28T17:58:54.999073+0000","last_became_active":"2024-03-28T17:58:54.005631+0000","last_became_peered":"2024-03-28T17:58:54.005631+0000","last_unstale":"2024-03-28T17:58:54.999073+0000","last_undegraded":"2024-03-28T17:58:54.999073+0000","last_fullsized":"2024-03-28T17:58:54.999073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000894+0000","last_change":"2024-03-28T17:58:54.005717+0000","last_active":"2024-03-28T17:58:55.000894+0000","last_peered":"2024-03-28T17:58:55.000894+0000","last_clean":"2024-03-28T17:58:55.000894+0000","last_became_active":"2024-03-28T17:58:54.005397+0000","last_became_peered":"2024-03-28T17:58:54.005397+0000","last_unstale":"2024-03-28T17:58:55.000894+0000","last_undegraded":"2024-03-28T17:58:55.000894+0000","last_fullsized":"2024-03-28T17:58:55.000894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997043+0000","last_change":"2024-03-28T17:58:54.009673+0000","last_active":"2024-03-28T17:58:54.997043+0000","last_peered":"2024-03-28T17:58:54.997043+0000","last_clean":"2024-03-28T17:58:54.997043+0000","last_became_active":"2024-03-28T17:58:54.009387+0000","last_became_peered":"2024-03-28T17:58:54.009387+0000","last_unstale":"2024-03-28T17:58:54.997043+0000","last_undegraded":"2024-03-28T17:58:54.997043+0000","last_fullsized":"2024-03-28T17:58:54.997043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:52.016086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095794+0000","last_change":"2024-03-28T17:58:54.003074+0000","last_active":"2024-03-28T17:58:55.095794+0000","last_peered":"2024-03-28T17:58:55.095794+0000","last_clean":"2024-03-28T17:58:55.095794+0000","last_became_active":"2024-03-28T17:58:54.002854+0000","last_became_peered":"2024-03-28T17:58:54.002854+0000","last_unstale":"2024-03-28T17:58:55.095794+0000","last_undegraded":"2024-03-28T17:58:55.095794+0000","last_fullsized":"2024-03-28T17:58:55.095794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019743+0000","last_change":"2024-03-28T17:58:54.022729+0000","last_active":"2024-03-28T17:58:55.019743+0000","last_peered":"2024-03-28T17:58:55.019743+0000","last_clean":"2024-03-28T17:58:55.019743+0000","last_became_active":"2024-03-28T17:58:54.022377+0000","last_became_peered":"2024-03-28T17:58:54.022377+0000","last_unstale":"2024-03-28T17:58:55.019743+0000","last_undegraded":"2024-03-28T17:58:55.019743+0000","last_fullsized":"2024-03-28T17:58:55.019743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098073+0000","last_change":"2024-03-28T17:58:54.005642+0000","last_active":"2024-03-28T17:58:55.098073+0000","last_peered":"2024-03-28T17:58:55.098073+0000","last_clean":"2024-03-28T17:58:55.098073+0000","last_became_active":"2024-03-28T17:58:54.005344+0000","last_became_peered":"2024-03-28T17:58:54.005344+0000","last_unstale":"2024-03-28T17:58:55.098073+0000","last_undegraded":"2024-03-28T17:58:55.098073+0000","last_fullsized":"2024-03-28T17:58:55.098073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999309+0000","last_change":"2024-03-28T17:58:54.005142+0000","last_active":"2024-03-28T17:58:54.999309+0000","last_peered":"2024-03-28T17:58:54.999309+0000","last_clean":"2024-03-28T17:58:54.999309+0000","last_became_active":"2024-03-28T17:58:54.003593+0000","last_became_peered":"2024-03-28T17:58:54.003593+0000","last_unstale":"2024-03-28T17:58:54.999309+0000","last_undegraded":"2024-03-28T17:58:54.999309+0000","last_fullsized":"2024-03-28T17:58:54.999309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095680+0000","last_change":"2024-03-28T17:58:54.020402+0000","last_active":"2024-03-28T17:58:55.095680+0000","last_peered":"2024-03-28T17:58:55.095680+0000","last_clean":"2024-03-28T17:58:55.095680+0000","last_became_active":"2024-03-28T17:58:54.020055+0000","last_became_peered":"2024-03-28T17:58:54.020055+0000","last_unstale":"2024-03-28T17:58:55.095680+0000","last_undegraded":"2024-03-28T17:58:55.095680+0000","last_fullsized":"2024-03-28T17:58:55.095680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999282+0000","last_change":"2024-03-28T17:58:54.012462+0000","last_active":"2024-03-28T17:58:54.999282+0000","last_peered":"2024-03-28T17:58:54.999282+0000","last_clean":"2024-03-28T17:58:54.999282+0000","last_became_active":"2024-03-28T17:58:54.011568+0000","last_became_peered":"2024-03-28T17:58:54.011568+0000","last_unstale":"2024-03-28T17:58:54.999282+0000","last_undegraded":"2024-03-28T17:58:54.999282+0000","last_fullsized":"2024-03-28T17:58:54.999282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095340+0000","last_change":"2024-03-28T17:58:54.023228+0000","last_active":"2024-03-28T17:58:55.095340+0000","last_peered":"2024-03-28T17:58:55.095340+0000","last_clean":"2024-03-28T17:58:55.095340+0000","last_became_active":"2024-03-28T17:58:54.022910+0000","last_became_peered":"2024-03-28T17:58:54.022910+0000","last_unstale":"2024-03-28T17:58:55.095340+0000","last_undegraded":"2024-03-28T17:58:55.095340+0000","last_fullsized":"2024-03-28T17:58:55.095340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019938+0000","last_change":"2024-03-28T17:58:54.006053+0000","last_active":"2024-03-28T17:58:55.019938+0000","last_peered":"2024-03-28T17:58:55.019938+0000","last_clean":"2024-03-28T17:58:55.019938+0000","last_became_active":"2024-03-28T17:58:54.005693+0000","last_became_peered":"2024-03-28T17:58:54.005693+0000","last_unstale":"2024-03-28T17:58:55.019938+0000","last_undegraded":"2024-03-28T17:58:55.019938+0000","last_fullsized":"2024-03-28T17:58:55.019938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998856+0000","last_change":"2024-03-28T17:58:54.013535+0000","last_active":"2024-03-28T17:58:54.998856+0000","last_peered":"2024-03-28T17:58:54.998856+0000","last_clean":"2024-03-28T17:58:54.998856+0000","last_became_active":"2024-03-28T17:58:54.011843+0000","last_became_peered":"2024-03-28T17:58:54.011843+0000","last_unstale":"2024-03-28T17:58:54.998856+0000","last_undegraded":"2024-03-28T17:58:54.998856+0000","last_fullsized":"2024-03-28T17:58:54.998856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095519+0000","last_change":"2024-03-28T17:58:54.022600+0000","last_active":"2024-03-28T17:58:55.095519+0000","last_peered":"2024-03-28T17:58:55.095519+0000","last_clean":"2024-03-28T17:58:55.095519+0000","last_became_active":"2024-03-28T17:58:54.022354+0000","last_became_peered":"2024-03-28T17:58:54.022354+0000","last_unstale":"2024-03-28T17:58:55.095519+0000","last_undegraded":"2024-03-28T17:58:55.095519+0000","last_fullsized":"2024-03-28T17:58:55.095519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097778+0000","last_change":"2024-03-28T17:58:54.005723+0000","last_active":"2024-03-28T17:58:55.097778+0000","last_peered":"2024-03-28T17:58:55.097778+0000","last_clean":"2024-03-28T17:58:55.097778+0000","last_became_active":"2024-03-28T17:58:54.005383+0000","last_became_peered":"2024-03-28T17:58:54.005383+0000","last_unstale":"2024-03-28T17:58:55.097778+0000","last_undegraded":"2024-03-28T17:58:55.097778+0000","last_fullsized":"2024-03-28T17:58:55.097778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:54.012921+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:54.012471+0000","last_became_peered":"2024-03-28T17:58:54.012471+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996217+0000","last_change":"2024-03-28T17:58:53.996340+0000","last_active":"2024-03-28T17:58:54.996217+0000","last_peered":"2024-03-28T17:58:54.996217+0000","last_clean":"2024-03-28T17:58:54.996217+0000","last_became_active":"2024-03-28T17:58:53.995914+0000","last_became_peered":"2024-03-28T17:58:53.995914+0000","last_unstale":"2024-03-28T17:58:54.996217+0000","last_undegraded":"2024-03-28T17:58:54.996217+0000","last_fullsized":"2024-03-28T17:58:54.996217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019333+0000","last_change":"2024-03-28T17:58:54.012238+0000","last_active":"2024-03-28T17:58:55.019333+0000","last_peered":"2024-03-28T17:58:55.019333+0000","last_clean":"2024-03-28T17:58:55.019333+0000","last_became_active":"2024-03-28T17:58:54.011952+0000","last_became_peered":"2024-03-28T17:58:54.011952+0000","last_unstale":"2024-03-28T17:58:55.019333+0000","last_undegraded":"2024-03-28T17:58:55.019333+0000","last_fullsized":"2024-03-28T17:58:55.019333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096509+0000","last_change":"2024-03-28T17:58:54.011087+0000","last_active":"2024-03-28T17:58:55.096509+0000","last_peered":"2024-03-28T17:58:55.096509+0000","last_clean":"2024-03-28T17:58:55.096509+0000","last_became_active":"2024-03-28T17:58:54.010787+0000","last_became_peered":"2024-03-28T17:58:54.010787+0000","last_unstale":"2024-03-28T17:58:55.096509+0000","last_undegraded":"2024-03-28T17:58:55.096509+0000","last_fullsized":"2024-03-28T17:58:55.096509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:23.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095155+0000","last_change":"2024-03-28T17:58:53.002153+0000","last_active":"2024-03-28T17:58:55.095155+0000","last_peered":"2024-03-28T17:58:55.095155+0000","last_clean":"2024-03-28T17:58:55.095155+0000","last_became_active":"2024-03-28T17:58:53.001844+0000","last_became_peered":"2024-03-28T17:58:53.001844+0000","last_unstale":"2024-03-28T17:58:55.095155+0000","last_undegraded":"2024-03-28T17:58:55.095155+0000","last_fullsized":"2024-03-28T17:58:55.095155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:38:14.392145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097946+0000","last_change":"2024-03-28T17:58:54.027047+0000","last_active":"2024-03-28T17:58:55.097946+0000","last_peered":"2024-03-28T17:58:55.097946+0000","last_clean":"2024-03-28T17:58:55.097946+0000","last_became_active":"2024-03-28T17:58:54.022997+0000","last_became_peered":"2024-03-28T17:58:54.022997+0000","last_unstale":"2024-03-28T17:58:55.097946+0000","last_undegraded":"2024-03-28T17:58:55.097946+0000","last_fullsized":"2024-03-28T17:58:55.097946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019270+0000","last_change":"2024-03-28T17:58:53.000389+0000","last_active":"2024-03-28T17:58:55.019270+0000","last_peered":"2024-03-28T17:58:55.019270+0000","last_clean":"2024-03-28T17:58:55.019270+0000","last_became_active":"2024-03-28T17:58:53.000086+0000","last_became_peered":"2024-03-28T17:58:53.000086+0000","last_unstale":"2024-03-28T17:58:55.019270+0000","last_undegraded":"2024-03-28T17:58:55.019270+0000","last_fullsized":"2024-03-28T17:58:55.019270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094817+0000","last_change":"2024-03-28T17:58:53.004667+0000","last_active":"2024-03-28T17:58:55.094817+0000","last_peered":"2024-03-28T17:58:55.094817+0000","last_clean":"2024-03-28T17:58:55.094817+0000","last_became_active":"2024-03-28T17:58:53.004337+0000","last_became_peered":"2024-03-28T17:58:53.004337+0000","last_unstale":"2024-03-28T17:58:55.094817+0000","last_undegraded":"2024-03-28T17:58:55.094817+0000","last_fullsized":"2024-03-28T17:58:55.094817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094868+0000","last_change":"2024-03-28T17:58:54.008588+0000","last_active":"2024-03-28T17:58:55.094868+0000","last_peered":"2024-03-28T17:58:55.094868+0000","last_clean":"2024-03-28T17:58:55.094868+0000","last_became_active":"2024-03-28T17:58:54.005942+0000","last_became_peered":"2024-03-28T17:58:54.005942+0000","last_unstale":"2024-03-28T17:58:55.094868+0000","last_undegraded":"2024-03-28T17:58:55.094868+0000","last_fullsized":"2024-03-28T17:58:55.094868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999261+0000","last_change":"2024-03-28T17:58:53.003611+0000","last_active":"2024-03-28T17:58:54.999261+0000","last_peered":"2024-03-28T17:58:54.999261+0000","last_clean":"2024-03-28T17:58:54.999261+0000","last_became_active":"2024-03-28T17:58:53.003272+0000","last_became_peered":"2024-03-28T17:58:53.003272+0000","last_unstale":"2024-03-28T17:58:54.999261+0000","last_undegraded":"2024-03-28T17:58:54.999261+0000","last_fullsized":"2024-03-28T17:58:54.999261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996592+0000","last_change":"2024-03-28T17:58:54.013586+0000","last_active":"2024-03-28T17:58:54.996592+0000","last_peered":"2024-03-28T17:58:54.996592+0000","last_clean":"2024-03-28T17:58:54.996592+0000","last_became_active":"2024-03-28T17:58:54.013293+0000","last_became_peered":"2024-03-28T17:58:54.013293+0000","last_unstale":"2024-03-28T17:58:54.996592+0000","last_undegraded":"2024-03-28T17:58:54.996592+0000","last_fullsized":"2024-03-28T17:58:54.996592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019821+0000","last_change":"2024-03-28T17:58:53.005607+0000","last_active":"2024-03-28T17:58:55.019821+0000","last_peered":"2024-03-28T17:58:55.019821+0000","last_clean":"2024-03-28T17:58:55.019821+0000","last_became_active":"2024-03-28T17:58:53.005306+0000","last_became_peered":"2024-03-28T17:58:53.005306+0000","last_unstale":"2024-03-28T17:58:55.019821+0000","last_undegraded":"2024-03-28T17:58:55.019821+0000","last_fullsized":"2024-03-28T17:58:55.019821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000404+0000","last_change":"2024-03-28T17:58:54.006871+0000","last_active":"2024-03-28T17:58:55.000404+0000","last_peered":"2024-03-28T17:58:55.000404+0000","last_clean":"2024-03-28T17:58:55.000404+0000","last_became_active":"2024-03-28T17:58:54.006521+0000","last_became_peered":"2024-03-28T17:58:54.006521+0000","last_unstale":"2024-03-28T17:58:55.000404+0000","last_undegraded":"2024-03-28T17:58:55.000404+0000","last_fullsized":"2024-03-28T17:58:55.000404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000147+0000","last_change":"2024-03-28T17:58:53.008708+0000","last_active":"2024-03-28T17:58:55.000147+0000","last_peered":"2024-03-28T17:58:55.000147+0000","last_clean":"2024-03-28T17:58:55.000147+0000","last_became_active":"2024-03-28T17:58:53.008424+0000","last_became_peered":"2024-03-28T17:58:53.008424+0000","last_unstale":"2024-03-28T17:58:55.000147+0000","last_undegraded":"2024-03-28T17:58:55.000147+0000","last_fullsized":"2024-03-28T17:58:55.000147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098103+0000","last_change":"2024-03-28T17:58:54.007645+0000","last_active":"2024-03-28T17:58:55.098103+0000","last_peered":"2024-03-28T17:58:55.098103+0000","last_clean":"2024-03-28T17:58:55.098103+0000","last_became_active":"2024-03-28T17:58:54.007359+0000","last_became_peered":"2024-03-28T17:58:54.007359+0000","last_unstale":"2024-03-28T17:58:55.098103+0000","last_undegraded":"2024-03-28T17:58:55.098103+0000","last_fullsized":"2024-03-28T17:58:55.098103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998573+0000","last_change":"2024-03-28T17:58:53.000532+0000","last_active":"2024-03-28T17:58:54.998573+0000","last_peered":"2024-03-28T17:58:54.998573+0000","last_clean":"2024-03-28T17:58:54.998573+0000","last_became_active":"2024-03-28T17:58:53.000238+0000","last_became_peered":"2024-03-28T17:58:53.000238+0000","last_unstale":"2024-03-28T17:58:54.998573+0000","last_undegraded":"2024-03-28T17:58:54.998573+0000","last_fullsized":"2024-03-28T17:58:54.998573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097269+0000","last_change":"2024-03-28T17:58:54.011919+0000","last_active":"2024-03-28T17:58:55.097269+0000","last_peered":"2024-03-28T17:58:55.097269+0000","last_clean":"2024-03-28T17:58:55.097269+0000","last_became_active":"2024-03-28T17:58:54.008256+0000","last_became_peered":"2024-03-28T17:58:54.008256+0000","last_unstale":"2024-03-28T17:58:55.097269+0000","last_undegraded":"2024-03-28T17:58:55.097269+0000","last_fullsized":"2024-03-28T17:58:55.097269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:16.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019656+0000","last_change":"2024-03-28T17:58:53.004395+0000","last_active":"2024-03-28T17:58:55.019656+0000","last_peered":"2024-03-28T17:58:55.019656+0000","last_clean":"2024-03-28T17:58:55.019656+0000","last_became_active":"2024-03-28T17:58:52.999296+0000","last_became_peered":"2024-03-28T17:58:52.999296+0000","last_unstale":"2024-03-28T17:58:55.019656+0000","last_undegraded":"2024-03-28T17:58:55.019656+0000","last_fullsized":"2024-03-28T17:58:55.019656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095473+0000","last_change":"2024-03-28T17:58:54.004875+0000","last_active":"2024-03-28T17:58:55.095473+0000","last_peered":"2024-03-28T17:58:55.095473+0000","last_clean":"2024-03-28T17:58:55.095473+0000","last_became_active":"2024-03-28T17:58:54.004597+0000","last_became_peered":"2024-03-28T17:58:54.004597+0000","last_unstale":"2024-03-28T17:58:55.095473+0000","last_undegraded":"2024-03-28T17:58:55.095473+0000","last_fullsized":"2024-03-28T17:58:55.095473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098023+0000","last_change":"2024-03-28T17:58:53.009461+0000","last_active":"2024-03-28T17:58:55.098023+0000","last_peered":"2024-03-28T17:58:55.098023+0000","last_clean":"2024-03-28T17:58:55.098023+0000","last_became_active":"2024-03-28T17:58:53.008867+0000","last_became_peered":"2024-03-28T17:58:53.008867+0000","last_unstale":"2024-03-28T17:58:55.098023+0000","last_undegraded":"2024-03-28T17:58:55.098023+0000","last_fullsized":"2024-03-28T17:58:55.098023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:57:02.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098575+0000","last_change":"2024-03-28T17:58:54.001368+0000","last_active":"2024-03-28T17:58:55.098575+0000","last_peered":"2024-03-28T17:58:55.098575+0000","last_clean":"2024-03-28T17:58:55.098575+0000","last_became_active":"2024-03-28T17:58:54.000744+0000","last_became_peered":"2024-03-28T17:58:54.000744+0000","last_unstale":"2024-03-28T17:58:55.098575+0000","last_undegraded":"2024-03-28T17:58:55.098575+0000","last_fullsized":"2024-03-28T17:58:55.098575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999461+0000","last_change":"2024-03-28T17:58:53.007374+0000","last_active":"2024-03-28T17:58:54.999461+0000","last_peered":"2024-03-28T17:58:54.999461+0000","last_clean":"2024-03-28T17:58:54.999461+0000","last_became_active":"2024-03-28T17:58:53.005001+0000","last_became_peered":"2024-03-28T17:58:53.005001+0000","last_unstale":"2024-03-28T17:58:54.999461+0000","last_undegraded":"2024-03-28T17:58:54.999461+0000","last_fullsized":"2024-03-28T17:58:54.999461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095283+0000","last_change":"2024-03-28T17:58:54.020789+0000","last_active":"2024-03-28T17:58:55.095283+0000","last_peered":"2024-03-28T17:58:55.095283+0000","last_clean":"2024-03-28T17:58:55.095283+0000","last_became_active":"2024-03-28T17:58:54.020338+0000","last_became_peered":"2024-03-28T17:58:54.020338+0000","last_unstale":"2024-03-28T17:58:55.095283+0000","last_undegraded":"2024-03-28T17:58:55.095283+0000","last_fullsized":"2024-03-28T17:58:55.095283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019218+0000","last_change":"2024-03-28T17:58:53.001274+0000","last_active":"2024-03-28T17:58:55.019218+0000","last_peered":"2024-03-28T17:58:55.019218+0000","last_clean":"2024-03-28T17:58:55.019218+0000","last_became_active":"2024-03-28T17:58:53.000949+0000","last_became_peered":"2024-03-28T17:58:53.000949+0000","last_unstale":"2024-03-28T17:58:55.019218+0000","last_undegraded":"2024-03-28T17:58:55.019218+0000","last_fullsized":"2024-03-28T17:58:55.019218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019704+0000","last_change":"2024-03-28T17:58:54.009630+0000","last_active":"2024-03-28T17:58:55.019704+0000","last_peered":"2024-03-28T17:58:55.019704+0000","last_clean":"2024-03-28T17:58:55.019704+0000","last_became_active":"2024-03-28T17:58:54.009072+0000","last_became_peered":"2024-03-28T17:58:54.009072+0000","last_unstale":"2024-03-28T17:58:55.019704+0000","last_undegraded":"2024-03-28T17:58:55.019704+0000","last_fullsized":"2024-03-28T17:58:55.019704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.995993+0000","last_change":"2024-03-28T17:58:53.002389+0000","last_active":"2024-03-28T17:58:54.995993+0000","last_peered":"2024-03-28T17:58:54.995993+0000","last_clean":"2024-03-28T17:58:54.995993+0000","last_became_active":"2024-03-28T17:58:53.001859+0000","last_became_peered":"2024-03-28T17:58:53.001859+0000","last_unstale":"2024-03-28T17:58:54.995993+0000","last_undegraded":"2024-03-28T17:58:54.995993+0000","last_fullsized":"2024-03-28T17:58:54.995993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095130+0000","last_change":"2024-03-28T17:58:54.022077+0000","last_active":"2024-03-28T17:58:55.095130+0000","last_peered":"2024-03-28T17:58:55.095130+0000","last_clean":"2024-03-28T17:58:55.095130+0000","last_became_active":"2024-03-28T17:58:54.021287+0000","last_became_peered":"2024-03-28T17:58:54.021287+0000","last_unstale":"2024-03-28T17:58:55.095130+0000","last_undegraded":"2024-03-28T17:58:55.095130+0000","last_fullsized":"2024-03-28T17:58:55.095130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097510+0000","last_change":"2024-03-28T17:58:53.002001+0000","last_active":"2024-03-28T17:58:55.097510+0000","last_peered":"2024-03-28T17:58:55.097510+0000","last_clean":"2024-03-28T17:58:55.097510+0000","last_became_active":"2024-03-28T17:58:53.001734+0000","last_became_peered":"2024-03-28T17:58:53.001734+0000","last_unstale":"2024-03-28T17:58:55.097510+0000","last_undegraded":"2024-03-28T17:58:55.097510+0000","last_fullsized":"2024-03-28T17:58:55.097510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:13:29.578775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095671+0000","last_change":"2024-03-28T17:58:54.021625+0000","last_active":"2024-03-28T17:58:55.095671+0000","last_peered":"2024-03-28T17:58:55.095671+0000","last_clean":"2024-03-28T17:58:55.095671+0000","last_became_active":"2024-03-28T17:58:54.020370+0000","last_became_peered":"2024-03-28T17:58:54.020370+0000","last_unstale":"2024-03-28T17:58:55.095671+0000","last_undegraded":"2024-03-28T17:58:55.095671+0000","last_fullsized":"2024-03-28T17:58:55.095671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997096+0000","last_change":"2024-03-28T17:58:52.997706+0000","last_active":"2024-03-28T17:58:54.997096+0000","last_peered":"2024-03-28T17:58:54.997096+0000","last_clean":"2024-03-28T17:58:54.997096+0000","last_became_active":"2024-03-28T17:58:52.997002+0000","last_became_peered":"2024-03-28T17:58:52.997002+0000","last_unstale":"2024-03-28T17:58:54.997096+0000","last_undegraded":"2024-03-28T17:58:54.997096+0000","last_fullsized":"2024-03-28T17:58:54.997096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096799+0000","last_change":"2024-03-28T17:58:54.021721+0000","last_active":"2024-03-28T17:58:55.096799+0000","last_peered":"2024-03-28T17:58:55.096799+0000","last_clean":"2024-03-28T17:58:55.096799+0000","last_became_active":"2024-03-28T17:58:54.020973+0000","last_became_peered":"2024-03-28T17:58:54.020973+0000","last_unstale":"2024-03-28T17:58:55.096799+0000","last_undegraded":"2024-03-28T17:58:55.096799+0000","last_fullsized":"2024-03-28T17:58:55.096799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:29:06.711377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998981+0000","last_change":"2024-03-28T17:58:53.007402+0000","last_active":"2024-03-28T17:58:54.998981+0000","last_peered":"2024-03-28T17:58:54.998981+0000","last_clean":"2024-03-28T17:58:54.998981+0000","last_became_active":"2024-03-28T17:58:53.007131+0000","last_became_peered":"2024-03-28T17:58:53.007131+0000","last_unstale":"2024-03-28T17:58:54.998981+0000","last_undegraded":"2024-03-28T17:58:54.998981+0000","last_fullsized":"2024-03-28T17:58:54.998981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000866+0000","last_change":"2024-03-28T17:58:54.009567+0000","last_active":"2024-03-28T17:58:55.000866+0000","last_peered":"2024-03-28T17:58:55.000866+0000","last_clean":"2024-03-28T17:58:55.000866+0000","last_became_active":"2024-03-28T17:58:54.008926+0000","last_became_peered":"2024-03-28T17:58:54.008926+0000","last_unstale":"2024-03-28T17:58:55.000866+0000","last_undegraded":"2024-03-28T17:58:55.000866+0000","last_fullsized":"2024-03-28T17:58:55.000866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996703+0000","last_change":"2024-03-28T17:58:52.995453+0000","last_active":"2024-03-28T17:58:54.996703+0000","last_peered":"2024-03-28T17:58:54.996703+0000","last_clean":"2024-03-28T17:58:54.996703+0000","last_became_active":"2024-03-28T17:58:52.995177+0000","last_became_peered":"2024-03-28T17:58:52.995177+0000","last_unstale":"2024-03-28T17:58:54.996703+0000","last_undegraded":"2024-03-28T17:58:54.996703+0000","last_fullsized":"2024-03-28T17:58:54.996703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095614+0000","last_change":"2024-03-28T17:58:54.004742+0000","last_active":"2024-03-28T17:58:55.095614+0000","last_peered":"2024-03-28T17:58:55.095614+0000","last_clean":"2024-03-28T17:58:55.095614+0000","last_became_active":"2024-03-28T17:58:54.004434+0000","last_became_peered":"2024-03-28T17:58:54.004434+0000","last_unstale":"2024-03-28T17:58:55.095614+0000","last_undegraded":"2024-03-28T17:58:55.095614+0000","last_fullsized":"2024-03-28T17:58:55.095614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094833+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T17:58:55.094833+0000","last_peered":"2024-03-28T17:58:55.094833+0000","last_clean":"2024-03-28T17:58:55.094833+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T17:58:55.094833+0000","last_undegraded":"2024-03-28T17:58:55.094833+0000","last_fullsized":"2024-03-28T17:58:55.094833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999042+0000","last_change":"2024-03-28T17:58:52.995644+0000","last_active":"2024-03-28T17:58:54.999042+0000","last_peered":"2024-03-28T17:58:54.999042+0000","last_clean":"2024-03-28T17:58:54.999042+0000","last_became_active":"2024-03-28T17:58:52.995196+0000","last_became_peered":"2024-03-28T17:58:52.995196+0000","last_unstale":"2024-03-28T17:58:54.999042+0000","last_undegraded":"2024-03-28T17:58:54.999042+0000","last_fullsized":"2024-03-28T17:58:54.999042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095066+0000","last_change":"2024-03-28T17:58:54.015875+0000","last_active":"2024-03-28T17:58:55.095066+0000","last_peered":"2024-03-28T17:58:55.095066+0000","last_clean":"2024-03-28T17:58:55.095066+0000","last_became_active":"2024-03-28T17:58:54.014187+0000","last_became_peered":"2024-03-28T17:58:54.014187+0000","last_unstale":"2024-03-28T17:58:55.095066+0000","last_undegraded":"2024-03-28T17:58:55.095066+0000","last_fullsized":"2024-03-28T17:58:55.095066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094699+0000","last_change":"2024-03-28T17:58:53.010263+0000","last_active":"2024-03-28T17:58:55.094699+0000","last_peered":"2024-03-28T17:58:55.094699+0000","last_clean":"2024-03-28T17:58:55.094699+0000","last_became_active":"2024-03-28T17:58:53.009990+0000","last_became_peered":"2024-03-28T17:58:53.009990+0000","last_unstale":"2024-03-28T17:58:55.094699+0000","last_undegraded":"2024-03-28T17:58:55.094699+0000","last_fullsized":"2024-03-28T17:58:55.094699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097226+0000","last_change":"2024-03-28T17:58:54.022349+0000","last_active":"2024-03-28T17:58:55.097226+0000","last_peered":"2024-03-28T17:58:55.097226+0000","last_clean":"2024-03-28T17:58:55.097226+0000","last_became_active":"2024-03-28T17:58:54.021997+0000","last_became_peered":"2024-03-28T17:58:54.021997+0000","last_unstale":"2024-03-28T17:58:55.097226+0000","last_undegraded":"2024-03-28T17:58:55.097226+0000","last_fullsized":"2024-03-28T17:58:55.097226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:20.314101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996301+0000","last_change":"2024-03-28T17:58:52.994588+0000","last_active":"2024-03-28T17:58:54.996301+0000","last_peered":"2024-03-28T17:58:54.996301+0000","last_clean":"2024-03-28T17:58:54.996301+0000","last_became_active":"2024-03-28T17:58:52.994338+0000","last_became_peered":"2024-03-28T17:58:52.994338+0000","last_unstale":"2024-03-28T17:58:54.996301+0000","last_undegraded":"2024-03-28T17:58:54.996301+0000","last_fullsized":"2024-03-28T17:58:54.996301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997461+0000","last_change":"2024-03-28T17:58:53.998446+0000","last_active":"2024-03-28T17:58:54.997461+0000","last_peered":"2024-03-28T17:58:54.997461+0000","last_clean":"2024-03-28T17:58:54.997461+0000","last_became_active":"2024-03-28T17:58:53.998163+0000","last_became_peered":"2024-03-28T17:58:53.998163+0000","last_unstale":"2024-03-28T17:58:54.997461+0000","last_undegraded":"2024-03-28T17:58:54.997461+0000","last_fullsized":"2024-03-28T17:58:54.997461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:52.998174+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:52.997265+0000","last_became_peered":"2024-03-28T17:58:52.997265+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096877+0000","last_change":"2024-03-28T17:58:54.017676+0000","last_active":"2024-03-28T17:58:55.096877+0000","last_peered":"2024-03-28T17:58:55.096877+0000","last_clean":"2024-03-28T17:58:55.096877+0000","last_became_active":"2024-03-28T17:58:54.016900+0000","last_became_peered":"2024-03-28T17:58:54.016900+0000","last_unstale":"2024-03-28T17:58:55.096877+0000","last_undegraded":"2024-03-28T17:58:55.096877+0000","last_fullsized":"2024-03-28T17:58:55.096877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:45.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998525+0000","last_change":"2024-03-28T17:58:52.997517+0000","last_active":"2024-03-28T17:58:54.998525+0000","last_peered":"2024-03-28T17:58:54.998525+0000","last_clean":"2024-03-28T17:58:54.998525+0000","last_became_active":"2024-03-28T17:58:52.997207+0000","last_became_peered":"2024-03-28T17:58:52.997207+0000","last_unstale":"2024-03-28T17:58:54.998525+0000","last_undegraded":"2024-03-28T17:58:54.998525+0000","last_fullsized":"2024-03-28T17:58:54.998525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999951+0000","last_change":"2024-03-28T17:58:54.011090+0000","last_active":"2024-03-28T17:58:54.999951+0000","last_peered":"2024-03-28T17:58:54.999951+0000","last_clean":"2024-03-28T17:58:54.999951+0000","last_became_active":"2024-03-28T17:58:54.009864+0000","last_became_peered":"2024-03-28T17:58:54.009864+0000","last_unstale":"2024-03-28T17:58:54.999951+0000","last_undegraded":"2024-03-28T17:58:54.999951+0000","last_fullsized":"2024-03-28T17:58:54.999951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095825+0000","last_change":"2024-03-28T17:58:53.011734+0000","last_active":"2024-03-28T17:58:55.095825+0000","last_peered":"2024-03-28T17:58:55.095825+0000","last_clean":"2024-03-28T17:58:55.095825+0000","last_became_active":"2024-03-28T17:58:53.011209+0000","last_became_peered":"2024-03-28T17:58:53.011209+0000","last_unstale":"2024-03-28T17:58:55.095825+0000","last_undegraded":"2024-03-28T17:58:55.095825+0000","last_fullsized":"2024-03-28T17:58:55.095825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997836+0000","last_change":"2024-03-28T17:58:54.015424+0000","last_active":"2024-03-28T17:58:54.997836+0000","last_peered":"2024-03-28T17:58:54.997836+0000","last_clean":"2024-03-28T17:58:54.997836+0000","last_became_active":"2024-03-28T17:58:54.011122+0000","last_became_peered":"2024-03-28T17:58:54.011122+0000","last_unstale":"2024-03-28T17:58:54.997836+0000","last_undegraded":"2024-03-28T17:58:54.997836+0000","last_fullsized":"2024-03-28T17:58:54.997836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999477+0000","last_change":"2024-03-28T17:58:53.008757+0000","last_active":"2024-03-28T17:58:54.999477+0000","last_peered":"2024-03-28T17:58:54.999477+0000","last_clean":"2024-03-28T17:58:54.999477+0000","last_became_active":"2024-03-28T17:58:53.008455+0000","last_became_peered":"2024-03-28T17:58:53.008455+0000","last_unstale":"2024-03-28T17:58:54.999477+0000","last_undegraded":"2024-03-28T17:58:54.999477+0000","last_fullsized":"2024-03-28T17:58:54.999477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096368+0000","last_change":"2024-03-28T17:58:54.017029+0000","last_active":"2024-03-28T17:58:55.096368+0000","last_peered":"2024-03-28T17:58:55.096368+0000","last_clean":"2024-03-28T17:58:55.096368+0000","last_became_active":"2024-03-28T17:58:54.014826+0000","last_became_peered":"2024-03-28T17:58:54.014826+0000","last_unstale":"2024-03-28T17:58:55.096368+0000","last_undegraded":"2024-03-28T17:58:55.096368+0000","last_fullsized":"2024-03-28T17:58:55.096368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019932+0000","last_change":"2024-03-28T17:58:53.005570+0000","last_active":"2024-03-28T17:58:55.019932+0000","last_peered":"2024-03-28T17:58:55.019932+0000","last_clean":"2024-03-28T17:58:55.019932+0000","last_became_active":"2024-03-28T17:58:53.005285+0000","last_became_peered":"2024-03-28T17:58:53.005285+0000","last_unstale":"2024-03-28T17:58:55.019932+0000","last_undegraded":"2024-03-28T17:58:55.019932+0000","last_fullsized":"2024-03-28T17:58:55.019932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098116+0000","last_change":"2024-03-28T17:58:54.018755+0000","last_active":"2024-03-28T17:58:55.098116+0000","last_peered":"2024-03-28T17:58:55.098116+0000","last_clean":"2024-03-28T17:58:55.098116+0000","last_became_active":"2024-03-28T17:58:54.018403+0000","last_became_peered":"2024-03-28T17:58:54.018403+0000","last_unstale":"2024-03-28T17:58:55.098116+0000","last_undegraded":"2024-03-28T17:58:55.098116+0000","last_fullsized":"2024-03-28T17:58:55.098116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095055+0000","last_change":"2024-03-28T17:58:53.008509+0000","last_active":"2024-03-28T17:58:55.095055+0000","last_peered":"2024-03-28T17:58:55.095055+0000","last_clean":"2024-03-28T17:58:55.095055+0000","last_became_active":"2024-03-28T17:58:53.008065+0000","last_became_peered":"2024-03-28T17:58:53.008065+0000","last_unstale":"2024-03-28T17:58:55.095055+0000","last_undegraded":"2024-03-28T17:58:55.095055+0000","last_fullsized":"2024-03-28T17:58:55.095055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095224+0000","last_change":"2024-03-28T17:58:54.018790+0000","last_active":"2024-03-28T17:58:55.095224+0000","last_peered":"2024-03-28T17:58:55.095224+0000","last_clean":"2024-03-28T17:58:55.095224+0000","last_became_active":"2024-03-28T17:58:54.018306+0000","last_became_peered":"2024-03-28T17:58:54.018306+0000","last_unstale":"2024-03-28T17:58:55.095224+0000","last_undegraded":"2024-03-28T17:58:55.095224+0000","last_fullsized":"2024-03-28T17:58:55.095224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:02.245546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996892+0000","last_change":"2024-03-28T17:58:53.007176+0000","last_active":"2024-03-28T17:58:54.996892+0000","last_peered":"2024-03-28T17:58:54.996892+0000","last_clean":"2024-03-28T17:58:54.996892+0000","last_became_active":"2024-03-28T17:58:53.002876+0000","last_became_peered":"2024-03-28T17:58:53.002876+0000","last_unstale":"2024-03-28T17:58:54.996892+0000","last_undegraded":"2024-03-28T17:58:54.996892+0000","last_fullsized":"2024-03-28T17:58:54.996892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095113+0000","last_change":"2024-03-28T17:58:54.021263+0000","last_active":"2024-03-28T17:58:55.095113+0000","last_peered":"2024-03-28T17:58:55.095113+0000","last_clean":"2024-03-28T17:58:55.095113+0000","last_became_active":"2024-03-28T17:58:54.015718+0000","last_became_peered":"2024-03-28T17:58:54.015718+0000","last_unstale":"2024-03-28T17:58:55.095113+0000","last_undegraded":"2024-03-28T17:58:55.095113+0000","last_fullsized":"2024-03-28T17:58:55.095113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997526+0000","last_change":"2024-03-28T17:58:53.000138+0000","last_active":"2024-03-28T17:58:54.997526+0000","last_peered":"2024-03-28T17:58:54.997526+0000","last_clean":"2024-03-28T17:58:54.997526+0000","last_became_active":"2024-03-28T17:58:52.999805+0000","last_became_peered":"2024-03-28T17:58:52.999805+0000","last_unstale":"2024-03-28T17:58:54.997526+0000","last_undegraded":"2024-03-28T17:58:54.997526+0000","last_fullsized":"2024-03-28T17:58:54.997526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095084+0000","last_change":"2024-03-28T17:58:54.006123+0000","last_active":"2024-03-28T17:58:55.095084+0000","last_peered":"2024-03-28T17:58:55.095084+0000","last_clean":"2024-03-28T17:58:55.095084+0000","last_became_active":"2024-03-28T17:58:54.005714+0000","last_became_peered":"2024-03-28T17:58:54.005714+0000","last_unstale":"2024-03-28T17:58:55.095084+0000","last_undegraded":"2024-03-28T17:58:55.095084+0000","last_fullsized":"2024-03-28T17:58:55.095084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095450+0000","last_change":"2024-03-28T17:58:53.009871+0000","last_active":"2024-03-28T17:58:55.095450+0000","last_peered":"2024-03-28T17:58:55.095450+0000","last_clean":"2024-03-28T17:58:55.095450+0000","last_became_active":"2024-03-28T17:58:53.009550+0000","last_became_peered":"2024-03-28T17:58:53.009550+0000","last_unstale":"2024-03-28T17:58:55.095450+0000","last_undegraded":"2024-03-28T17:58:55.095450+0000","last_fullsized":"2024-03-28T17:58:55.095450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:03.353429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019393+0000","last_change":"2024-03-28T17:58:54.020986+0000","last_active":"2024-03-28T17:58:55.019393+0000","last_peered":"2024-03-28T17:58:55.019393+0000","last_clean":"2024-03-28T17:58:55.019393+0000","last_became_active":"2024-03-28T17:58:54.020717+0000","last_became_peered":"2024-03-28T17:58:54.020717+0000","last_unstale":"2024-03-28T17:58:55.019393+0000","last_undegraded":"2024-03-28T17:58:55.019393+0000","last_fullsized":"2024-03-28T17:58:55.019393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999845+0000","last_change":"2024-03-28T17:58:54.022957+0000","last_active":"2024-03-28T17:58:54.999845+0000","last_peered":"2024-03-28T17:58:54.999845+0000","last_clean":"2024-03-28T17:58:54.999845+0000","last_became_active":"2024-03-28T17:58:54.022234+0000","last_became_peered":"2024-03-28T17:58:54.022234+0000","last_unstale":"2024-03-28T17:58:54.999845+0000","last_undegraded":"2024-03-28T17:58:54.999845+0000","last_fullsized":"2024-03-28T17:58:54.999845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097078+0000","last_change":"2024-03-28T17:58:53.010194+0000","last_active":"2024-03-28T17:58:55.097078+0000","last_peered":"2024-03-28T17:58:55.097078+0000","last_clean":"2024-03-28T17:58:55.097078+0000","last_became_active":"2024-03-28T17:58:53.009750+0000","last_became_peered":"2024-03-28T17:58:53.009750+0000","last_unstale":"2024-03-28T17:58:55.097078+0000","last_undegraded":"2024-03-28T17:58:55.097078+0000","last_fullsized":"2024-03-28T17:58:55.097078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000545+0000","last_change":"2024-03-28T17:58:54.016961+0000","last_active":"2024-03-28T17:58:55.000545+0000","last_peered":"2024-03-28T17:58:55.000545+0000","last_clean":"2024-03-28T17:58:55.000545+0000","last_became_active":"2024-03-28T17:58:54.016695+0000","last_became_peered":"2024-03-28T17:58:54.016695+0000","last_unstale":"2024-03-28T17:58:55.000545+0000","last_undegraded":"2024-03-28T17:58:55.000545+0000","last_fullsized":"2024-03-28T17:58:55.000545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095647+0000","last_change":"2024-03-28T17:58:53.011900+0000","last_active":"2024-03-28T17:58:55.095647+0000","last_peered":"2024-03-28T17:58:55.095647+0000","last_clean":"2024-03-28T17:58:55.095647+0000","last_became_active":"2024-03-28T17:58:53.011463+0000","last_became_peered":"2024-03-28T17:58:53.011463+0000","last_unstale":"2024-03-28T17:58:55.095647+0000","last_undegraded":"2024-03-28T17:58:55.095647+0000","last_fullsized":"2024-03-28T17:58:55.095647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098095+0000","last_change":"2024-03-28T17:58:54.010789+0000","last_active":"2024-03-28T17:58:55.098095+0000","last_peered":"2024-03-28T17:58:55.098095+0000","last_clean":"2024-03-28T17:58:55.098095+0000","last_became_active":"2024-03-28T17:58:54.010520+0000","last_became_peered":"2024-03-28T17:58:54.010520+0000","last_unstale":"2024-03-28T17:58:55.098095+0000","last_undegraded":"2024-03-28T17:58:55.098095+0000","last_fullsized":"2024-03-28T17:58:55.098095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095653+0000","last_change":"2024-03-28T17:58:52.999999+0000","last_active":"2024-03-28T17:58:55.095653+0000","last_peered":"2024-03-28T17:58:55.095653+0000","last_clean":"2024-03-28T17:58:55.095653+0000","last_became_active":"2024-03-28T17:58:52.999659+0000","last_became_peered":"2024-03-28T17:58:52.999659+0000","last_unstale":"2024-03-28T17:58:55.095653+0000","last_undegraded":"2024-03-28T17:58:55.095653+0000","last_fullsized":"2024-03-28T17:58:55.095653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095619+0000","last_change":"2024-03-28T17:58:54.010224+0000","last_active":"2024-03-28T17:58:55.095619+0000","last_peered":"2024-03-28T17:58:55.095619+0000","last_clean":"2024-03-28T17:58:55.095619+0000","last_became_active":"2024-03-28T17:58:54.008396+0000","last_became_peered":"2024-03-28T17:58:54.008396+0000","last_unstale":"2024-03-28T17:58:55.095619+0000","last_undegraded":"2024-03-28T17:58:55.095619+0000","last_fullsized":"2024-03-28T17:58:55.095619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098248+0000","last_change":"2024-03-28T17:58:53.008301+0000","last_active":"2024-03-28T17:58:55.098248+0000","last_peered":"2024-03-28T17:58:55.098248+0000","last_clean":"2024-03-28T17:58:55.098248+0000","last_became_active":"2024-03-28T17:58:53.007920+0000","last_became_peered":"2024-03-28T17:58:53.007920+0000","last_unstale":"2024-03-28T17:58:55.098248+0000","last_undegraded":"2024-03-28T17:58:55.098248+0000","last_fullsized":"2024-03-28T17:58:55.098248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T17:59:04.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T17:59:04.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.406+0000 7f9c69200640 1 Processor -- start 2024-03-28T17:59:04.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.406+0000 7f9c69200640 1 -- start start 2024-03-28T17:59:04.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c69200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c6415b9c0 0x7f9c640717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c69200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9c640722b0 0x7f9c64072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c69200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c64070a40 con 0x7f9c6406caa0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c69200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c64071330 con 0x7f9c6415b9c0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c69200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c640b3af0 con 0x7f9c640722b0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c63400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c6415b9c0 0x7f9c640717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c63e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9c6406caa0 0x7f9c6406cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53696/0 (socket says 172.21.15.62:53696) 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c63e00640 1 -- 172.21.15.62:0/1314483931 learned_addr learned my addr 172.21.15.62:0/1314483931 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.407+0000 7f9c5aa00640 1 --2- 172.21.15.62:0/1314483931 >> v2:172.21.15.81:3300/0 conn(0x7f9c640722b0 0x7f9c64072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.408+0000 7f9c62a00640 1 -- 172.21.15.62:0/1314483931 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292375223 0 0) 0x7f9c64070a40 con 0x7f9c6406caa0 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.408+0000 7f9c63400640 1 -- 172.21.15.62:0/1314483931 >> v2:172.21.15.81:3300/0 conn(0x7f9c640722b0 msgr2=0x7f9c64072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.408+0000 7f9c63400640 1 --2- 172.21.15.62:0/1314483931 >> v2:172.21.15.81:3300/0 conn(0x7f9c640722b0 0x7f9c64072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.408+0000 7f9c63400640 1 -- 172.21.15.62:0/1314483931 >> v1:172.21.15.62:6789/0 conn(0x7f9c6406caa0 legacy=0x7f9c6406cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.408+0000 7f9c63400640 1 -- 172.21.15.62:0/1314483931 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640b3810 con 0x7f9c6415b9c0 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.409+0000 7f9c63400640 1 --2- 172.21.15.62:0/1314483931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6415b9c0 0x7f9c640717c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c640af670 tx=0x7f9c5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32082d8725a9f4a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.409+0000 7f9c62a00640 1 -- 172.21.15.62:0/1314483931 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c540315a0 con 0x7f9c6415b9c0 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.409+0000 7f9c62a00640 1 -- 172.21.15.62:0/1314483931 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c54037070 con 0x7f9c6415b9c0 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.409+0000 7f9c62a00640 1 -- 172.21.15.62:0/1314483931 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54031960 con 0x7f9c6415b9c0 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 -- 172.21.15.62:0/1314483931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6415b9c0 msgr2=0x7f9c640717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 --2- 172.21.15.62:0/1314483931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6415b9c0 0x7f9c640717c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c640af670 tx=0x7f9c5402b840 comp rx=0 tx=0).stop 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 -- 172.21.15.62:0/1314483931 shutdown_connections 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 --2- 172.21.15.62:0/1314483931 >> v2:172.21.15.81:3300/0 conn(0x7f9c640722b0 0x7f9c64072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 --2- 172.21.15.62:0/1314483931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6415b9c0 0x7f9c640717c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 -- 172.21.15.62:0/1314483931 >> 172.21.15.62:0/1314483931 conn(0x7f9c640b1730 msgr2=0x7f9c640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 -- 172.21.15.62:0/1314483931 shutdown_connections 2024-03-28T17:59:04.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.410+0000 7f9c69200640 1 -- 172.21.15.62:0/1314483931 wait complete. 2024-03-28T17:59:04.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.411+0000 7f9c69200640 1 Processor -- start 2024-03-28T17:59:04.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 -- start start 2024-03-28T17:59:04.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640722b0 0x7f9c6415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6415b9c0 0x7f9c640cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415bda0 con 0x7f9c6406caa0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64070fc0 con 0x7f9c640722b0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.412+0000 7f9c69200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64071d00 con 0x7f9c6415b9c0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58188/0 (socket says 172.21.15.62:58188) 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 -- 172.21.15.62:0/3489605782 learned_addr learned my addr 172.21.15.62:0/3489605782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c5aa00640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640722b0 0x7f9c6415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6415b9c0 0x7f9c640cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 -- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6415b9c0 msgr2=0x7f9c640cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6415b9c0 0x7f9c640cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 -- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640722b0 msgr2=0x7f9c6415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640722b0 0x7f9c6415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 -- 172.21.15.62:0/3489605782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6414cab0 con 0x7f9c6406caa0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.413+0000 7f9c63400640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009890 tx=0x7f9c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.414+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54032630 con 0x7f9c6406caa0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.414+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c540327d0 con 0x7f9c6406caa0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.414+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54032ba0 con 0x7f9c6406caa0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.414+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c640cd030 con 0x7f9c6406caa0 2024-03-28T17:59:04.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.414+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640cd5f0 con 0x7f9c6406caa0 2024-03-28T17:59:04.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.415+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6406b370 con 0x7f9c6406caa0 2024-03-28T17:59:04.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.416+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c5404c020 con 0x7f9c6406caa0 2024-03-28T17:59:04.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.417+0000 7f9c5be00640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 0x7f9c4006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.417+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c5402f020 con 0x7f9c6406caa0 2024-03-28T17:59:04.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.417+0000 7f9c5aa00640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 0x7f9c4006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.418+0000 7f9c5aa00640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 0x7f9c4006e090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c6414c730 tx=0x7f9c4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.422+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9c6414cab0 con 0x7f9c6406caa0 2024-03-28T17:59:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.614+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c64073490 con 0x7f9c4006bbd0 2024-03-28T17:59:04.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.628+0000 7f9c5be00640 1 -- 172.21.15.62:0/3489605782 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251837 (secure 0 0 0) 0x7f9c64073490 con 0x7f9c4006bbd0 2024-03-28T17:59:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:04.634 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 msgr2=0x7f9c4006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 0x7f9c4006e090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c6414c730 tx=0x7f9c4c008040 comp rx=0 tx=0).stop 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 msgr2=0x7f9c6414c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009890 tx=0x7f9c540046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 shutdown_connections 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4006bbd0 0x7f9c4006e090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6415b9c0 0x7f9c640cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640722b0 0x7f9c6415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 --2- 172.21.15.62:0/3489605782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6406caa0 0x7f9c6414c1f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 >> 172.21.15.62:0/3489605782 conn(0x7f9c640b1730 msgr2=0x7f9c640b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:04.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 shutdown_connections 2024-03-28T17:59:04.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.634+0000 7f9c69200640 1 -- 172.21.15.62:0/3489605782 wait complete. 2024-03-28T17:59:04.651 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-28T17:59:04.307856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003724"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096023+0000","last_change":"2024-03-28T17:58:54.011100+0000","last_active":"2024-03-28T17:58:55.096023+0000","last_peered":"2024-03-28T17:58:55.096023+0000","last_clean":"2024-03-28T17:58:55.096023+0000","last_became_active":"2024-03-28T17:58:54.010769+0000","last_became_peered":"2024-03-28T17:58:54.010769+0000","last_unstale":"2024-03-28T17:58:55.096023+0000","last_undegraded":"2024-03-28T17:58:55.096023+0000","last_fullsized":"2024-03-28T17:58:55.096023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999288+0000","last_change":"2024-03-28T17:58:54.016056+0000","last_active":"2024-03-28T17:58:54.999288+0000","last_peered":"2024-03-28T17:58:54.999288+0000","last_clean":"2024-03-28T17:58:54.999288+0000","last_became_active":"2024-03-28T17:58:54.012351+0000","last_became_peered":"2024-03-28T17:58:54.012351+0000","last_unstale":"2024-03-28T17:58:54.999288+0000","last_undegraded":"2024-03-28T17:58:54.999288+0000","last_fullsized":"2024-03-28T17:58:54.999288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097234+0000","last_change":"2024-03-28T17:58:53.996524+0000","last_active":"2024-03-28T17:58:55.097234+0000","last_peered":"2024-03-28T17:58:55.097234+0000","last_clean":"2024-03-28T17:58:55.097234+0000","last_became_active":"2024-03-28T17:58:53.996197+0000","last_became_peered":"2024-03-28T17:58:53.996197+0000","last_unstale":"2024-03-28T17:58:55.097234+0000","last_undegraded":"2024-03-28T17:58:55.097234+0000","last_fullsized":"2024-03-28T17:58:55.097234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:16.038136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.001103+0000","last_change":"2024-03-28T17:58:54.009958+0000","last_active":"2024-03-28T17:58:55.001103+0000","last_peered":"2024-03-28T17:58:55.001103+0000","last_clean":"2024-03-28T17:58:55.001103+0000","last_became_active":"2024-03-28T17:58:54.009279+0000","last_became_peered":"2024-03-28T17:58:54.009279+0000","last_unstale":"2024-03-28T17:58:55.001103+0000","last_undegraded":"2024-03-28T17:58:55.001103+0000","last_fullsized":"2024-03-28T17:58:55.001103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095127+0000","last_change":"2024-03-28T17:58:54.007553+0000","last_active":"2024-03-28T17:58:55.095127+0000","last_peered":"2024-03-28T17:58:55.095127+0000","last_clean":"2024-03-28T17:58:55.095127+0000","last_became_active":"2024-03-28T17:58:54.007177+0000","last_became_peered":"2024-03-28T17:58:54.007177+0000","last_unstale":"2024-03-28T17:58:55.095127+0000","last_undegraded":"2024-03-28T17:58:55.095127+0000","last_fullsized":"2024-03-28T17:58:55.095127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999258+0000","last_change":"2024-03-28T17:58:53.998917+0000","last_active":"2024-03-28T17:58:54.999258+0000","last_peered":"2024-03-28T17:58:54.999258+0000","last_clean":"2024-03-28T17:58:54.999258+0000","last_became_active":"2024-03-28T17:58:53.998692+0000","last_became_peered":"2024-03-28T17:58:53.998692+0000","last_unstale":"2024-03-28T17:58:54.999258+0000","last_undegraded":"2024-03-28T17:58:54.999258+0000","last_fullsized":"2024-03-28T17:58:54.999258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094845+0000","last_change":"2024-03-28T17:58:54.007407+0000","last_active":"2024-03-28T17:58:55.094845+0000","last_peered":"2024-03-28T17:58:55.094845+0000","last_clean":"2024-03-28T17:58:55.094845+0000","last_became_active":"2024-03-28T17:58:54.007126+0000","last_became_peered":"2024-03-28T17:58:54.007126+0000","last_unstale":"2024-03-28T17:58:55.094845+0000","last_undegraded":"2024-03-28T17:58:55.094845+0000","last_fullsized":"2024-03-28T17:58:55.094845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094632+0000","last_change":"2024-03-28T17:58:54.021995+0000","last_active":"2024-03-28T17:58:55.094632+0000","last_peered":"2024-03-28T17:58:55.094632+0000","last_clean":"2024-03-28T17:58:55.094632+0000","last_became_active":"2024-03-28T17:58:54.021661+0000","last_became_peered":"2024-03-28T17:58:54.021661+0000","last_unstale":"2024-03-28T17:58:55.094632+0000","last_undegraded":"2024-03-28T17:58:55.094632+0000","last_fullsized":"2024-03-28T17:58:55.094632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096407+0000","last_change":"2024-03-28T17:58:54.014331+0000","last_active":"2024-03-28T17:58:55.096407+0000","last_peered":"2024-03-28T17:58:55.096407+0000","last_clean":"2024-03-28T17:58:55.096407+0000","last_became_active":"2024-03-28T17:58:54.014014+0000","last_became_peered":"2024-03-28T17:58:54.014014+0000","last_unstale":"2024-03-28T17:58:55.096407+0000","last_undegraded":"2024-03-28T17:58:55.096407+0000","last_fullsized":"2024-03-28T17:58:55.096407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:40:57.167073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019925+0000","last_change":"2024-03-28T17:58:54.013693+0000","last_active":"2024-03-28T17:58:55.019925+0000","last_peered":"2024-03-28T17:58:55.019925+0000","last_clean":"2024-03-28T17:58:55.019925+0000","last_became_active":"2024-03-28T17:58:54.013397+0000","last_became_peered":"2024-03-28T17:58:54.013397+0000","last_unstale":"2024-03-28T17:58:55.019925+0000","last_undegraded":"2024-03-28T17:58:55.019925+0000","last_fullsized":"2024-03-28T17:58:55.019925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019799+0000","last_change":"2024-03-28T17:58:54.014516+0000","last_active":"2024-03-28T17:58:55.019799+0000","last_peered":"2024-03-28T17:58:55.019799+0000","last_clean":"2024-03-28T17:58:55.019799+0000","last_became_active":"2024-03-28T17:58:54.014122+0000","last_became_peered":"2024-03-28T17:58:54.014122+0000","last_unstale":"2024-03-28T17:58:55.019799+0000","last_undegraded":"2024-03-28T17:58:55.019799+0000","last_fullsized":"2024-03-28T17:58:55.019799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097339+0000","last_change":"2024-03-28T17:58:54.015180+0000","last_active":"2024-03-28T17:58:55.097339+0000","last_peered":"2024-03-28T17:58:55.097339+0000","last_clean":"2024-03-28T17:58:55.097339+0000","last_became_active":"2024-03-28T17:58:54.008685+0000","last_became_peered":"2024-03-28T17:58:54.008685+0000","last_unstale":"2024-03-28T17:58:55.097339+0000","last_undegraded":"2024-03-28T17:58:55.097339+0000","last_fullsized":"2024-03-28T17:58:55.097339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095295+0000","last_change":"2024-03-28T17:58:54.012021+0000","last_active":"2024-03-28T17:58:55.095295+0000","last_peered":"2024-03-28T17:58:55.095295+0000","last_clean":"2024-03-28T17:58:55.095295+0000","last_became_active":"2024-03-28T17:58:54.011650+0000","last_became_peered":"2024-03-28T17:58:54.011650+0000","last_unstale":"2024-03-28T17:58:55.095295+0000","last_undegraded":"2024-03-28T17:58:55.095295+0000","last_fullsized":"2024-03-28T17:58:55.095295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097361+0000","last_change":"2024-03-28T17:58:54.011097+0000","last_active":"2024-03-28T17:58:55.097361+0000","last_peered":"2024-03-28T17:58:55.097361+0000","last_clean":"2024-03-28T17:58:55.097361+0000","last_became_active":"2024-03-28T17:58:54.010790+0000","last_became_peered":"2024-03-28T17:58:54.010790+0000","last_unstale":"2024-03-28T17:58:55.097361+0000","last_undegraded":"2024-03-28T17:58:55.097361+0000","last_fullsized":"2024-03-28T17:58:55.097361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:51:58.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996815+0000","last_change":"2024-03-28T17:58:54.022714+0000","last_active":"2024-03-28T17:58:54.996815+0000","last_peered":"2024-03-28T17:58:54.996815+0000","last_clean":"2024-03-28T17:58:54.996815+0000","last_became_active":"2024-03-28T17:58:54.022409+0000","last_became_peered":"2024-03-28T17:58:54.022409+0000","last_unstale":"2024-03-28T17:58:54.996815+0000","last_undegraded":"2024-03-28T17:58:54.996815+0000","last_fullsized":"2024-03-28T17:58:54.996815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999073+0000","last_change":"2024-03-28T17:58:54.005916+0000","last_active":"2024-03-28T17:58:54.999073+0000","last_peered":"2024-03-28T17:58:54.999073+0000","last_clean":"2024-03-28T17:58:54.999073+0000","last_became_active":"2024-03-28T17:58:54.005631+0000","last_became_peered":"2024-03-28T17:58:54.005631+0000","last_unstale":"2024-03-28T17:58:54.999073+0000","last_undegraded":"2024-03-28T17:58:54.999073+0000","last_fullsized":"2024-03-28T17:58:54.999073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000894+0000","last_change":"2024-03-28T17:58:54.005717+0000","last_active":"2024-03-28T17:58:55.000894+0000","last_peered":"2024-03-28T17:58:55.000894+0000","last_clean":"2024-03-28T17:58:55.000894+0000","last_became_active":"2024-03-28T17:58:54.005397+0000","last_became_peered":"2024-03-28T17:58:54.005397+0000","last_unstale":"2024-03-28T17:58:55.000894+0000","last_undegraded":"2024-03-28T17:58:55.000894+0000","last_fullsized":"2024-03-28T17:58:55.000894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:12.248610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997043+0000","last_change":"2024-03-28T17:58:54.009673+0000","last_active":"2024-03-28T17:58:54.997043+0000","last_peered":"2024-03-28T17:58:54.997043+0000","last_clean":"2024-03-28T17:58:54.997043+0000","last_became_active":"2024-03-28T17:58:54.009387+0000","last_became_peered":"2024-03-28T17:58:54.009387+0000","last_unstale":"2024-03-28T17:58:54.997043+0000","last_undegraded":"2024-03-28T17:58:54.997043+0000","last_fullsized":"2024-03-28T17:58:54.997043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:52.016086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095794+0000","last_change":"2024-03-28T17:58:54.003074+0000","last_active":"2024-03-28T17:58:55.095794+0000","last_peered":"2024-03-28T17:58:55.095794+0000","last_clean":"2024-03-28T17:58:55.095794+0000","last_became_active":"2024-03-28T17:58:54.002854+0000","last_became_peered":"2024-03-28T17:58:54.002854+0000","last_unstale":"2024-03-28T17:58:55.095794+0000","last_undegraded":"2024-03-28T17:58:55.095794+0000","last_fullsized":"2024-03-28T17:58:55.095794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019743+0000","last_change":"2024-03-28T17:58:54.022729+0000","last_active":"2024-03-28T17:58:55.019743+0000","last_peered":"2024-03-28T17:58:55.019743+0000","last_clean":"2024-03-28T17:58:55.019743+0000","last_became_active":"2024-03-28T17:58:54.022377+0000","last_became_peered":"2024-03-28T17:58:54.022377+0000","last_unstale":"2024-03-28T17:58:55.019743+0000","last_undegraded":"2024-03-28T17:58:55.019743+0000","last_fullsized":"2024-03-28T17:58:55.019743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098073+0000","last_change":"2024-03-28T17:58:54.005642+0000","last_active":"2024-03-28T17:58:55.098073+0000","last_peered":"2024-03-28T17:58:55.098073+0000","last_clean":"2024-03-28T17:58:55.098073+0000","last_became_active":"2024-03-28T17:58:54.005344+0000","last_became_peered":"2024-03-28T17:58:54.005344+0000","last_unstale":"2024-03-28T17:58:55.098073+0000","last_undegraded":"2024-03-28T17:58:55.098073+0000","last_fullsized":"2024-03-28T17:58:55.098073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999309+0000","last_change":"2024-03-28T17:58:54.005142+0000","last_active":"2024-03-28T17:58:54.999309+0000","last_peered":"2024-03-28T17:58:54.999309+0000","last_clean":"2024-03-28T17:58:54.999309+0000","last_became_active":"2024-03-28T17:58:54.003593+0000","last_became_peered":"2024-03-28T17:58:54.003593+0000","last_unstale":"2024-03-28T17:58:54.999309+0000","last_undegraded":"2024-03-28T17:58:54.999309+0000","last_fullsized":"2024-03-28T17:58:54.999309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095680+0000","last_change":"2024-03-28T17:58:54.020402+0000","last_active":"2024-03-28T17:58:55.095680+0000","last_peered":"2024-03-28T17:58:55.095680+0000","last_clean":"2024-03-28T17:58:55.095680+0000","last_became_active":"2024-03-28T17:58:54.020055+0000","last_became_peered":"2024-03-28T17:58:54.020055+0000","last_unstale":"2024-03-28T17:58:55.095680+0000","last_undegraded":"2024-03-28T17:58:55.095680+0000","last_fullsized":"2024-03-28T17:58:55.095680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999282+0000","last_change":"2024-03-28T17:58:54.012462+0000","last_active":"2024-03-28T17:58:54.999282+0000","last_peered":"2024-03-28T17:58:54.999282+0000","last_clean":"2024-03-28T17:58:54.999282+0000","last_became_active":"2024-03-28T17:58:54.011568+0000","last_became_peered":"2024-03-28T17:58:54.011568+0000","last_unstale":"2024-03-28T17:58:54.999282+0000","last_undegraded":"2024-03-28T17:58:54.999282+0000","last_fullsized":"2024-03-28T17:58:54.999282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095340+0000","last_change":"2024-03-28T17:58:54.023228+0000","last_active":"2024-03-28T17:58:55.095340+0000","last_peered":"2024-03-28T17:58:55.095340+0000","last_clean":"2024-03-28T17:58:55.095340+0000","last_became_active":"2024-03-28T17:58:54.022910+0000","last_became_peered":"2024-03-28T17:58:54.022910+0000","last_unstale":"2024-03-28T17:58:55.095340+0000","last_undegraded":"2024-03-28T17:58:55.095340+0000","last_fullsized":"2024-03-28T17:58:55.095340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019938+0000","last_change":"2024-03-28T17:58:54.006053+0000","last_active":"2024-03-28T17:58:55.019938+0000","last_peered":"2024-03-28T17:58:55.019938+0000","last_clean":"2024-03-28T17:58:55.019938+0000","last_became_active":"2024-03-28T17:58:54.005693+0000","last_became_peered":"2024-03-28T17:58:54.005693+0000","last_unstale":"2024-03-28T17:58:55.019938+0000","last_undegraded":"2024-03-28T17:58:55.019938+0000","last_fullsized":"2024-03-28T17:58:55.019938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998856+0000","last_change":"2024-03-28T17:58:54.013535+0000","last_active":"2024-03-28T17:58:54.998856+0000","last_peered":"2024-03-28T17:58:54.998856+0000","last_clean":"2024-03-28T17:58:54.998856+0000","last_became_active":"2024-03-28T17:58:54.011843+0000","last_became_peered":"2024-03-28T17:58:54.011843+0000","last_unstale":"2024-03-28T17:58:54.998856+0000","last_undegraded":"2024-03-28T17:58:54.998856+0000","last_fullsized":"2024-03-28T17:58:54.998856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095519+0000","last_change":"2024-03-28T17:58:54.022600+0000","last_active":"2024-03-28T17:58:55.095519+0000","last_peered":"2024-03-28T17:58:55.095519+0000","last_clean":"2024-03-28T17:58:55.095519+0000","last_became_active":"2024-03-28T17:58:54.022354+0000","last_became_peered":"2024-03-28T17:58:54.022354+0000","last_unstale":"2024-03-28T17:58:55.095519+0000","last_undegraded":"2024-03-28T17:58:55.095519+0000","last_fullsized":"2024-03-28T17:58:55.095519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:33.509412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097778+0000","last_change":"2024-03-28T17:58:54.005723+0000","last_active":"2024-03-28T17:58:55.097778+0000","last_peered":"2024-03-28T17:58:55.097778+0000","last_clean":"2024-03-28T17:58:55.097778+0000","last_became_active":"2024-03-28T17:58:54.005383+0000","last_became_peered":"2024-03-28T17:58:54.005383+0000","last_unstale":"2024-03-28T17:58:55.097778+0000","last_undegraded":"2024-03-28T17:58:55.097778+0000","last_fullsized":"2024-03-28T17:58:55.097778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:54.012921+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:54.012471+0000","last_became_peered":"2024-03-28T17:58:54.012471+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:46.951433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996217+0000","last_change":"2024-03-28T17:58:53.996340+0000","last_active":"2024-03-28T17:58:54.996217+0000","last_peered":"2024-03-28T17:58:54.996217+0000","last_clean":"2024-03-28T17:58:54.996217+0000","last_became_active":"2024-03-28T17:58:53.995914+0000","last_became_peered":"2024-03-28T17:58:53.995914+0000","last_unstale":"2024-03-28T17:58:54.996217+0000","last_undegraded":"2024-03-28T17:58:54.996217+0000","last_fullsized":"2024-03-28T17:58:54.996217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:01.120420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019333+0000","last_change":"2024-03-28T17:58:54.012238+0000","last_active":"2024-03-28T17:58:55.019333+0000","last_peered":"2024-03-28T17:58:55.019333+0000","last_clean":"2024-03-28T17:58:55.019333+0000","last_became_active":"2024-03-28T17:58:54.011952+0000","last_became_peered":"2024-03-28T17:58:54.011952+0000","last_unstale":"2024-03-28T17:58:55.019333+0000","last_undegraded":"2024-03-28T17:58:55.019333+0000","last_fullsized":"2024-03-28T17:58:55.019333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:49.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096509+0000","last_change":"2024-03-28T17:58:54.011087+0000","last_active":"2024-03-28T17:58:55.096509+0000","last_peered":"2024-03-28T17:58:55.096509+0000","last_clean":"2024-03-28T17:58:55.096509+0000","last_became_active":"2024-03-28T17:58:54.010787+0000","last_became_peered":"2024-03-28T17:58:54.010787+0000","last_unstale":"2024-03-28T17:58:55.096509+0000","last_undegraded":"2024-03-28T17:58:55.096509+0000","last_fullsized":"2024-03-28T17:58:55.096509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:23.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095155+0000","last_change":"2024-03-28T17:58:53.002153+0000","last_active":"2024-03-28T17:58:55.095155+0000","last_peered":"2024-03-28T17:58:55.095155+0000","last_clean":"2024-03-28T17:58:55.095155+0000","last_became_active":"2024-03-28T17:58:53.001844+0000","last_became_peered":"2024-03-28T17:58:53.001844+0000","last_unstale":"2024-03-28T17:58:55.095155+0000","last_undegraded":"2024-03-28T17:58:55.095155+0000","last_fullsized":"2024-03-28T17:58:55.095155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:38:14.392145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097946+0000","last_change":"2024-03-28T17:58:54.027047+0000","last_active":"2024-03-28T17:58:55.097946+0000","last_peered":"2024-03-28T17:58:55.097946+0000","last_clean":"2024-03-28T17:58:55.097946+0000","last_became_active":"2024-03-28T17:58:54.022997+0000","last_became_peered":"2024-03-28T17:58:54.022997+0000","last_unstale":"2024-03-28T17:58:55.097946+0000","last_undegraded":"2024-03-28T17:58:55.097946+0000","last_fullsized":"2024-03-28T17:58:55.097946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019270+0000","last_change":"2024-03-28T17:58:53.000389+0000","last_active":"2024-03-28T17:58:55.019270+0000","last_peered":"2024-03-28T17:58:55.019270+0000","last_clean":"2024-03-28T17:58:55.019270+0000","last_became_active":"2024-03-28T17:58:53.000086+0000","last_became_peered":"2024-03-28T17:58:53.000086+0000","last_unstale":"2024-03-28T17:58:55.019270+0000","last_undegraded":"2024-03-28T17:58:55.019270+0000","last_fullsized":"2024-03-28T17:58:55.019270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094817+0000","last_change":"2024-03-28T17:58:53.004667+0000","last_active":"2024-03-28T17:58:55.094817+0000","last_peered":"2024-03-28T17:58:55.094817+0000","last_clean":"2024-03-28T17:58:55.094817+0000","last_became_active":"2024-03-28T17:58:53.004337+0000","last_became_peered":"2024-03-28T17:58:53.004337+0000","last_unstale":"2024-03-28T17:58:55.094817+0000","last_undegraded":"2024-03-28T17:58:55.094817+0000","last_fullsized":"2024-03-28T17:58:55.094817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094868+0000","last_change":"2024-03-28T17:58:54.008588+0000","last_active":"2024-03-28T17:58:55.094868+0000","last_peered":"2024-03-28T17:58:55.094868+0000","last_clean":"2024-03-28T17:58:55.094868+0000","last_became_active":"2024-03-28T17:58:54.005942+0000","last_became_peered":"2024-03-28T17:58:54.005942+0000","last_unstale":"2024-03-28T17:58:55.094868+0000","last_undegraded":"2024-03-28T17:58:55.094868+0000","last_fullsized":"2024-03-28T17:58:55.094868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999261+0000","last_change":"2024-03-28T17:58:53.003611+0000","last_active":"2024-03-28T17:58:54.999261+0000","last_peered":"2024-03-28T17:58:54.999261+0000","last_clean":"2024-03-28T17:58:54.999261+0000","last_became_active":"2024-03-28T17:58:53.003272+0000","last_became_peered":"2024-03-28T17:58:53.003272+0000","last_unstale":"2024-03-28T17:58:54.999261+0000","last_undegraded":"2024-03-28T17:58:54.999261+0000","last_fullsized":"2024-03-28T17:58:54.999261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996592+0000","last_change":"2024-03-28T17:58:54.013586+0000","last_active":"2024-03-28T17:58:54.996592+0000","last_peered":"2024-03-28T17:58:54.996592+0000","last_clean":"2024-03-28T17:58:54.996592+0000","last_became_active":"2024-03-28T17:58:54.013293+0000","last_became_peered":"2024-03-28T17:58:54.013293+0000","last_unstale":"2024-03-28T17:58:54.996592+0000","last_undegraded":"2024-03-28T17:58:54.996592+0000","last_fullsized":"2024-03-28T17:58:54.996592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019821+0000","last_change":"2024-03-28T17:58:53.005607+0000","last_active":"2024-03-28T17:58:55.019821+0000","last_peered":"2024-03-28T17:58:55.019821+0000","last_clean":"2024-03-28T17:58:55.019821+0000","last_became_active":"2024-03-28T17:58:53.005306+0000","last_became_peered":"2024-03-28T17:58:53.005306+0000","last_unstale":"2024-03-28T17:58:55.019821+0000","last_undegraded":"2024-03-28T17:58:55.019821+0000","last_fullsized":"2024-03-28T17:58:55.019821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000404+0000","last_change":"2024-03-28T17:58:54.006871+0000","last_active":"2024-03-28T17:58:55.000404+0000","last_peered":"2024-03-28T17:58:55.000404+0000","last_clean":"2024-03-28T17:58:55.000404+0000","last_became_active":"2024-03-28T17:58:54.006521+0000","last_became_peered":"2024-03-28T17:58:54.006521+0000","last_unstale":"2024-03-28T17:58:55.000404+0000","last_undegraded":"2024-03-28T17:58:55.000404+0000","last_fullsized":"2024-03-28T17:58:55.000404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000147+0000","last_change":"2024-03-28T17:58:53.008708+0000","last_active":"2024-03-28T17:58:55.000147+0000","last_peered":"2024-03-28T17:58:55.000147+0000","last_clean":"2024-03-28T17:58:55.000147+0000","last_became_active":"2024-03-28T17:58:53.008424+0000","last_became_peered":"2024-03-28T17:58:53.008424+0000","last_unstale":"2024-03-28T17:58:55.000147+0000","last_undegraded":"2024-03-28T17:58:55.000147+0000","last_fullsized":"2024-03-28T17:58:55.000147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098103+0000","last_change":"2024-03-28T17:58:54.007645+0000","last_active":"2024-03-28T17:58:55.098103+0000","last_peered":"2024-03-28T17:58:55.098103+0000","last_clean":"2024-03-28T17:58:55.098103+0000","last_became_active":"2024-03-28T17:58:54.007359+0000","last_became_peered":"2024-03-28T17:58:54.007359+0000","last_unstale":"2024-03-28T17:58:55.098103+0000","last_undegraded":"2024-03-28T17:58:55.098103+0000","last_fullsized":"2024-03-28T17:58:55.098103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998573+0000","last_change":"2024-03-28T17:58:53.000532+0000","last_active":"2024-03-28T17:58:54.998573+0000","last_peered":"2024-03-28T17:58:54.998573+0000","last_clean":"2024-03-28T17:58:54.998573+0000","last_became_active":"2024-03-28T17:58:53.000238+0000","last_became_peered":"2024-03-28T17:58:53.000238+0000","last_unstale":"2024-03-28T17:58:54.998573+0000","last_undegraded":"2024-03-28T17:58:54.998573+0000","last_fullsized":"2024-03-28T17:58:54.998573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097269+0000","last_change":"2024-03-28T17:58:54.011919+0000","last_active":"2024-03-28T17:58:55.097269+0000","last_peered":"2024-03-28T17:58:55.097269+0000","last_clean":"2024-03-28T17:58:55.097269+0000","last_became_active":"2024-03-28T17:58:54.008256+0000","last_became_peered":"2024-03-28T17:58:54.008256+0000","last_unstale":"2024-03-28T17:58:55.097269+0000","last_undegraded":"2024-03-28T17:58:55.097269+0000","last_fullsized":"2024-03-28T17:58:55.097269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:16.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019656+0000","last_change":"2024-03-28T17:58:53.004395+0000","last_active":"2024-03-28T17:58:55.019656+0000","last_peered":"2024-03-28T17:58:55.019656+0000","last_clean":"2024-03-28T17:58:55.019656+0000","last_became_active":"2024-03-28T17:58:52.999296+0000","last_became_peered":"2024-03-28T17:58:52.999296+0000","last_unstale":"2024-03-28T17:58:55.019656+0000","last_undegraded":"2024-03-28T17:58:55.019656+0000","last_fullsized":"2024-03-28T17:58:55.019656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095473+0000","last_change":"2024-03-28T17:58:54.004875+0000","last_active":"2024-03-28T17:58:55.095473+0000","last_peered":"2024-03-28T17:58:55.095473+0000","last_clean":"2024-03-28T17:58:55.095473+0000","last_became_active":"2024-03-28T17:58:54.004597+0000","last_became_peered":"2024-03-28T17:58:54.004597+0000","last_unstale":"2024-03-28T17:58:55.095473+0000","last_undegraded":"2024-03-28T17:58:55.095473+0000","last_fullsized":"2024-03-28T17:58:55.095473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098023+0000","last_change":"2024-03-28T17:58:53.009461+0000","last_active":"2024-03-28T17:58:55.098023+0000","last_peered":"2024-03-28T17:58:55.098023+0000","last_clean":"2024-03-28T17:58:55.098023+0000","last_became_active":"2024-03-28T17:58:53.008867+0000","last_became_peered":"2024-03-28T17:58:53.008867+0000","last_unstale":"2024-03-28T17:58:55.098023+0000","last_undegraded":"2024-03-28T17:58:55.098023+0000","last_fullsized":"2024-03-28T17:58:55.098023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:57:02.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098575+0000","last_change":"2024-03-28T17:58:54.001368+0000","last_active":"2024-03-28T17:58:55.098575+0000","last_peered":"2024-03-28T17:58:55.098575+0000","last_clean":"2024-03-28T17:58:55.098575+0000","last_became_active":"2024-03-28T17:58:54.000744+0000","last_became_peered":"2024-03-28T17:58:54.000744+0000","last_unstale":"2024-03-28T17:58:55.098575+0000","last_undegraded":"2024-03-28T17:58:55.098575+0000","last_fullsized":"2024-03-28T17:58:55.098575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:17.152965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999461+0000","last_change":"2024-03-28T17:58:53.007374+0000","last_active":"2024-03-28T17:58:54.999461+0000","last_peered":"2024-03-28T17:58:54.999461+0000","last_clean":"2024-03-28T17:58:54.999461+0000","last_became_active":"2024-03-28T17:58:53.005001+0000","last_became_peered":"2024-03-28T17:58:53.005001+0000","last_unstale":"2024-03-28T17:58:54.999461+0000","last_undegraded":"2024-03-28T17:58:54.999461+0000","last_fullsized":"2024-03-28T17:58:54.999461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095283+0000","last_change":"2024-03-28T17:58:54.020789+0000","last_active":"2024-03-28T17:58:55.095283+0000","last_peered":"2024-03-28T17:58:55.095283+0000","last_clean":"2024-03-28T17:58:55.095283+0000","last_became_active":"2024-03-28T17:58:54.020338+0000","last_became_peered":"2024-03-28T17:58:54.020338+0000","last_unstale":"2024-03-28T17:58:55.095283+0000","last_undegraded":"2024-03-28T17:58:55.095283+0000","last_fullsized":"2024-03-28T17:58:55.095283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019218+0000","last_change":"2024-03-28T17:58:53.001274+0000","last_active":"2024-03-28T17:58:55.019218+0000","last_peered":"2024-03-28T17:58:55.019218+0000","last_clean":"2024-03-28T17:58:55.019218+0000","last_became_active":"2024-03-28T17:58:53.000949+0000","last_became_peered":"2024-03-28T17:58:53.000949+0000","last_unstale":"2024-03-28T17:58:55.019218+0000","last_undegraded":"2024-03-28T17:58:55.019218+0000","last_fullsized":"2024-03-28T17:58:55.019218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019704+0000","last_change":"2024-03-28T17:58:54.009630+0000","last_active":"2024-03-28T17:58:55.019704+0000","last_peered":"2024-03-28T17:58:55.019704+0000","last_clean":"2024-03-28T17:58:55.019704+0000","last_became_active":"2024-03-28T17:58:54.009072+0000","last_became_peered":"2024-03-28T17:58:54.009072+0000","last_unstale":"2024-03-28T17:58:55.019704+0000","last_undegraded":"2024-03-28T17:58:55.019704+0000","last_fullsized":"2024-03-28T17:58:55.019704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.995993+0000","last_change":"2024-03-28T17:58:53.002389+0000","last_active":"2024-03-28T17:58:54.995993+0000","last_peered":"2024-03-28T17:58:54.995993+0000","last_clean":"2024-03-28T17:58:54.995993+0000","last_became_active":"2024-03-28T17:58:53.001859+0000","last_became_peered":"2024-03-28T17:58:53.001859+0000","last_unstale":"2024-03-28T17:58:54.995993+0000","last_undegraded":"2024-03-28T17:58:54.995993+0000","last_fullsized":"2024-03-28T17:58:54.995993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095130+0000","last_change":"2024-03-28T17:58:54.022077+0000","last_active":"2024-03-28T17:58:55.095130+0000","last_peered":"2024-03-28T17:58:55.095130+0000","last_clean":"2024-03-28T17:58:55.095130+0000","last_became_active":"2024-03-28T17:58:54.021287+0000","last_became_peered":"2024-03-28T17:58:54.021287+0000","last_unstale":"2024-03-28T17:58:55.095130+0000","last_undegraded":"2024-03-28T17:58:55.095130+0000","last_fullsized":"2024-03-28T17:58:55.095130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097510+0000","last_change":"2024-03-28T17:58:53.002001+0000","last_active":"2024-03-28T17:58:55.097510+0000","last_peered":"2024-03-28T17:58:55.097510+0000","last_clean":"2024-03-28T17:58:55.097510+0000","last_became_active":"2024-03-28T17:58:53.001734+0000","last_became_peered":"2024-03-28T17:58:53.001734+0000","last_unstale":"2024-03-28T17:58:55.097510+0000","last_undegraded":"2024-03-28T17:58:55.097510+0000","last_fullsized":"2024-03-28T17:58:55.097510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:13:29.578775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095671+0000","last_change":"2024-03-28T17:58:54.021625+0000","last_active":"2024-03-28T17:58:55.095671+0000","last_peered":"2024-03-28T17:58:55.095671+0000","last_clean":"2024-03-28T17:58:55.095671+0000","last_became_active":"2024-03-28T17:58:54.020370+0000","last_became_peered":"2024-03-28T17:58:54.020370+0000","last_unstale":"2024-03-28T17:58:55.095671+0000","last_undegraded":"2024-03-28T17:58:55.095671+0000","last_fullsized":"2024-03-28T17:58:55.095671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997096+0000","last_change":"2024-03-28T17:58:52.997706+0000","last_active":"2024-03-28T17:58:54.997096+0000","last_peered":"2024-03-28T17:58:54.997096+0000","last_clean":"2024-03-28T17:58:54.997096+0000","last_became_active":"2024-03-28T17:58:52.997002+0000","last_became_peered":"2024-03-28T17:58:52.997002+0000","last_unstale":"2024-03-28T17:58:54.997096+0000","last_undegraded":"2024-03-28T17:58:54.997096+0000","last_fullsized":"2024-03-28T17:58:54.997096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096799+0000","last_change":"2024-03-28T17:58:54.021721+0000","last_active":"2024-03-28T17:58:55.096799+0000","last_peered":"2024-03-28T17:58:55.096799+0000","last_clean":"2024-03-28T17:58:55.096799+0000","last_became_active":"2024-03-28T17:58:54.020973+0000","last_became_peered":"2024-03-28T17:58:54.020973+0000","last_unstale":"2024-03-28T17:58:55.096799+0000","last_undegraded":"2024-03-28T17:58:55.096799+0000","last_fullsized":"2024-03-28T17:58:55.096799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:29:06.711377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998981+0000","last_change":"2024-03-28T17:58:53.007402+0000","last_active":"2024-03-28T17:58:54.998981+0000","last_peered":"2024-03-28T17:58:54.998981+0000","last_clean":"2024-03-28T17:58:54.998981+0000","last_became_active":"2024-03-28T17:58:53.007131+0000","last_became_peered":"2024-03-28T17:58:53.007131+0000","last_unstale":"2024-03-28T17:58:54.998981+0000","last_undegraded":"2024-03-28T17:58:54.998981+0000","last_fullsized":"2024-03-28T17:58:54.998981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000866+0000","last_change":"2024-03-28T17:58:54.009567+0000","last_active":"2024-03-28T17:58:55.000866+0000","last_peered":"2024-03-28T17:58:55.000866+0000","last_clean":"2024-03-28T17:58:55.000866+0000","last_became_active":"2024-03-28T17:58:54.008926+0000","last_became_peered":"2024-03-28T17:58:54.008926+0000","last_unstale":"2024-03-28T17:58:55.000866+0000","last_undegraded":"2024-03-28T17:58:55.000866+0000","last_fullsized":"2024-03-28T17:58:55.000866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:44.257413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996703+0000","last_change":"2024-03-28T17:58:52.995453+0000","last_active":"2024-03-28T17:58:54.996703+0000","last_peered":"2024-03-28T17:58:54.996703+0000","last_clean":"2024-03-28T17:58:54.996703+0000","last_became_active":"2024-03-28T17:58:52.995177+0000","last_became_peered":"2024-03-28T17:58:52.995177+0000","last_unstale":"2024-03-28T17:58:54.996703+0000","last_undegraded":"2024-03-28T17:58:54.996703+0000","last_fullsized":"2024-03-28T17:58:54.996703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095614+0000","last_change":"2024-03-28T17:58:54.004742+0000","last_active":"2024-03-28T17:58:55.095614+0000","last_peered":"2024-03-28T17:58:55.095614+0000","last_clean":"2024-03-28T17:58:55.095614+0000","last_became_active":"2024-03-28T17:58:54.004434+0000","last_became_peered":"2024-03-28T17:58:54.004434+0000","last_unstale":"2024-03-28T17:58:55.095614+0000","last_undegraded":"2024-03-28T17:58:55.095614+0000","last_fullsized":"2024-03-28T17:58:55.095614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:12:01.372802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094833+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T17:58:55.094833+0000","last_peered":"2024-03-28T17:58:55.094833+0000","last_clean":"2024-03-28T17:58:55.094833+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T17:58:55.094833+0000","last_undegraded":"2024-03-28T17:58:55.094833+0000","last_fullsized":"2024-03-28T17:58:55.094833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999042+0000","last_change":"2024-03-28T17:58:52.995644+0000","last_active":"2024-03-28T17:58:54.999042+0000","last_peered":"2024-03-28T17:58:54.999042+0000","last_clean":"2024-03-28T17:58:54.999042+0000","last_became_active":"2024-03-28T17:58:52.995196+0000","last_became_peered":"2024-03-28T17:58:52.995196+0000","last_unstale":"2024-03-28T17:58:54.999042+0000","last_undegraded":"2024-03-28T17:58:54.999042+0000","last_fullsized":"2024-03-28T17:58:54.999042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:40.440966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095066+0000","last_change":"2024-03-28T17:58:54.015875+0000","last_active":"2024-03-28T17:58:55.095066+0000","last_peered":"2024-03-28T17:58:55.095066+0000","last_clean":"2024-03-28T17:58:55.095066+0000","last_became_active":"2024-03-28T17:58:54.014187+0000","last_became_peered":"2024-03-28T17:58:54.014187+0000","last_unstale":"2024-03-28T17:58:55.095066+0000","last_undegraded":"2024-03-28T17:58:55.095066+0000","last_fullsized":"2024-03-28T17:58:55.095066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:43:01.174976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.094699+0000","last_change":"2024-03-28T17:58:53.010263+0000","last_active":"2024-03-28T17:58:55.094699+0000","last_peered":"2024-03-28T17:58:55.094699+0000","last_clean":"2024-03-28T17:58:55.094699+0000","last_became_active":"2024-03-28T17:58:53.009990+0000","last_became_peered":"2024-03-28T17:58:53.009990+0000","last_unstale":"2024-03-28T17:58:55.094699+0000","last_undegraded":"2024-03-28T17:58:55.094699+0000","last_fullsized":"2024-03-28T17:58:55.094699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097226+0000","last_change":"2024-03-28T17:58:54.022349+0000","last_active":"2024-03-28T17:58:55.097226+0000","last_peered":"2024-03-28T17:58:55.097226+0000","last_clean":"2024-03-28T17:58:55.097226+0000","last_became_active":"2024-03-28T17:58:54.021997+0000","last_became_peered":"2024-03-28T17:58:54.021997+0000","last_unstale":"2024-03-28T17:58:55.097226+0000","last_undegraded":"2024-03-28T17:58:55.097226+0000","last_fullsized":"2024-03-28T17:58:55.097226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:29:20.314101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996301+0000","last_change":"2024-03-28T17:58:52.994588+0000","last_active":"2024-03-28T17:58:54.996301+0000","last_peered":"2024-03-28T17:58:54.996301+0000","last_clean":"2024-03-28T17:58:54.996301+0000","last_became_active":"2024-03-28T17:58:52.994338+0000","last_became_peered":"2024-03-28T17:58:52.994338+0000","last_unstale":"2024-03-28T17:58:54.996301+0000","last_undegraded":"2024-03-28T17:58:54.996301+0000","last_fullsized":"2024-03-28T17:58:54.996301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997461+0000","last_change":"2024-03-28T17:58:53.998446+0000","last_active":"2024-03-28T17:58:54.997461+0000","last_peered":"2024-03-28T17:58:54.997461+0000","last_clean":"2024-03-28T17:58:54.997461+0000","last_became_active":"2024-03-28T17:58:53.998163+0000","last_became_peered":"2024-03-28T17:58:53.998163+0000","last_unstale":"2024-03-28T17:58:54.997461+0000","last_undegraded":"2024-03-28T17:58:54.997461+0000","last_fullsized":"2024-03-28T17:58:54.997461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999759+0000","last_change":"2024-03-28T17:58:52.998174+0000","last_active":"2024-03-28T17:58:54.999759+0000","last_peered":"2024-03-28T17:58:54.999759+0000","last_clean":"2024-03-28T17:58:54.999759+0000","last_became_active":"2024-03-28T17:58:52.997265+0000","last_became_peered":"2024-03-28T17:58:52.997265+0000","last_unstale":"2024-03-28T17:58:54.999759+0000","last_undegraded":"2024-03-28T17:58:54.999759+0000","last_fullsized":"2024-03-28T17:58:54.999759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096877+0000","last_change":"2024-03-28T17:58:54.017676+0000","last_active":"2024-03-28T17:58:55.096877+0000","last_peered":"2024-03-28T17:58:55.096877+0000","last_clean":"2024-03-28T17:58:55.096877+0000","last_became_active":"2024-03-28T17:58:54.016900+0000","last_became_peered":"2024-03-28T17:58:54.016900+0000","last_unstale":"2024-03-28T17:58:55.096877+0000","last_undegraded":"2024-03-28T17:58:55.096877+0000","last_fullsized":"2024-03-28T17:58:55.096877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:45.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.998525+0000","last_change":"2024-03-28T17:58:52.997517+0000","last_active":"2024-03-28T17:58:54.998525+0000","last_peered":"2024-03-28T17:58:54.998525+0000","last_clean":"2024-03-28T17:58:54.998525+0000","last_became_active":"2024-03-28T17:58:52.997207+0000","last_became_peered":"2024-03-28T17:58:52.997207+0000","last_unstale":"2024-03-28T17:58:54.998525+0000","last_undegraded":"2024-03-28T17:58:54.998525+0000","last_fullsized":"2024-03-28T17:58:54.998525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999951+0000","last_change":"2024-03-28T17:58:54.011090+0000","last_active":"2024-03-28T17:58:54.999951+0000","last_peered":"2024-03-28T17:58:54.999951+0000","last_clean":"2024-03-28T17:58:54.999951+0000","last_became_active":"2024-03-28T17:58:54.009864+0000","last_became_peered":"2024-03-28T17:58:54.009864+0000","last_unstale":"2024-03-28T17:58:54.999951+0000","last_undegraded":"2024-03-28T17:58:54.999951+0000","last_fullsized":"2024-03-28T17:58:54.999951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:06.616838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095825+0000","last_change":"2024-03-28T17:58:53.011734+0000","last_active":"2024-03-28T17:58:55.095825+0000","last_peered":"2024-03-28T17:58:55.095825+0000","last_clean":"2024-03-28T17:58:55.095825+0000","last_became_active":"2024-03-28T17:58:53.011209+0000","last_became_peered":"2024-03-28T17:58:53.011209+0000","last_unstale":"2024-03-28T17:58:55.095825+0000","last_undegraded":"2024-03-28T17:58:55.095825+0000","last_fullsized":"2024-03-28T17:58:55.095825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:00.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997836+0000","last_change":"2024-03-28T17:58:54.015424+0000","last_active":"2024-03-28T17:58:54.997836+0000","last_peered":"2024-03-28T17:58:54.997836+0000","last_clean":"2024-03-28T17:58:54.997836+0000","last_became_active":"2024-03-28T17:58:54.011122+0000","last_became_peered":"2024-03-28T17:58:54.011122+0000","last_unstale":"2024-03-28T17:58:54.997836+0000","last_undegraded":"2024-03-28T17:58:54.997836+0000","last_fullsized":"2024-03-28T17:58:54.997836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999477+0000","last_change":"2024-03-28T17:58:53.008757+0000","last_active":"2024-03-28T17:58:54.999477+0000","last_peered":"2024-03-28T17:58:54.999477+0000","last_clean":"2024-03-28T17:58:54.999477+0000","last_became_active":"2024-03-28T17:58:53.008455+0000","last_became_peered":"2024-03-28T17:58:53.008455+0000","last_unstale":"2024-03-28T17:58:54.999477+0000","last_undegraded":"2024-03-28T17:58:54.999477+0000","last_fullsized":"2024-03-28T17:58:54.999477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.096368+0000","last_change":"2024-03-28T17:58:54.017029+0000","last_active":"2024-03-28T17:58:55.096368+0000","last_peered":"2024-03-28T17:58:55.096368+0000","last_clean":"2024-03-28T17:58:55.096368+0000","last_became_active":"2024-03-28T17:58:54.014826+0000","last_became_peered":"2024-03-28T17:58:54.014826+0000","last_unstale":"2024-03-28T17:58:55.096368+0000","last_undegraded":"2024-03-28T17:58:55.096368+0000","last_fullsized":"2024-03-28T17:58:55.096368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019932+0000","last_change":"2024-03-28T17:58:53.005570+0000","last_active":"2024-03-28T17:58:55.019932+0000","last_peered":"2024-03-28T17:58:55.019932+0000","last_clean":"2024-03-28T17:58:55.019932+0000","last_became_active":"2024-03-28T17:58:53.005285+0000","last_became_peered":"2024-03-28T17:58:53.005285+0000","last_unstale":"2024-03-28T17:58:55.019932+0000","last_undegraded":"2024-03-28T17:58:55.019932+0000","last_fullsized":"2024-03-28T17:58:55.019932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098116+0000","last_change":"2024-03-28T17:58:54.018755+0000","last_active":"2024-03-28T17:58:55.098116+0000","last_peered":"2024-03-28T17:58:55.098116+0000","last_clean":"2024-03-28T17:58:55.098116+0000","last_became_active":"2024-03-28T17:58:54.018403+0000","last_became_peered":"2024-03-28T17:58:54.018403+0000","last_unstale":"2024-03-28T17:58:55.098116+0000","last_undegraded":"2024-03-28T17:58:55.098116+0000","last_fullsized":"2024-03-28T17:58:55.098116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:36.990357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095055+0000","last_change":"2024-03-28T17:58:53.008509+0000","last_active":"2024-03-28T17:58:55.095055+0000","last_peered":"2024-03-28T17:58:55.095055+0000","last_clean":"2024-03-28T17:58:55.095055+0000","last_became_active":"2024-03-28T17:58:53.008065+0000","last_became_peered":"2024-03-28T17:58:53.008065+0000","last_unstale":"2024-03-28T17:58:55.095055+0000","last_undegraded":"2024-03-28T17:58:55.095055+0000","last_fullsized":"2024-03-28T17:58:55.095055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:22.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095224+0000","last_change":"2024-03-28T17:58:54.018790+0000","last_active":"2024-03-28T17:58:55.095224+0000","last_peered":"2024-03-28T17:58:55.095224+0000","last_clean":"2024-03-28T17:58:55.095224+0000","last_became_active":"2024-03-28T17:58:54.018306+0000","last_became_peered":"2024-03-28T17:58:54.018306+0000","last_unstale":"2024-03-28T17:58:55.095224+0000","last_undegraded":"2024-03-28T17:58:55.095224+0000","last_fullsized":"2024-03-28T17:58:55.095224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:02.245546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.996892+0000","last_change":"2024-03-28T17:58:53.007176+0000","last_active":"2024-03-28T17:58:54.996892+0000","last_peered":"2024-03-28T17:58:54.996892+0000","last_clean":"2024-03-28T17:58:54.996892+0000","last_became_active":"2024-03-28T17:58:53.002876+0000","last_became_peered":"2024-03-28T17:58:53.002876+0000","last_unstale":"2024-03-28T17:58:54.996892+0000","last_undegraded":"2024-03-28T17:58:54.996892+0000","last_fullsized":"2024-03-28T17:58:54.996892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:01.230401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095113+0000","last_change":"2024-03-28T17:58:54.021263+0000","last_active":"2024-03-28T17:58:55.095113+0000","last_peered":"2024-03-28T17:58:55.095113+0000","last_clean":"2024-03-28T17:58:55.095113+0000","last_became_active":"2024-03-28T17:58:54.015718+0000","last_became_peered":"2024-03-28T17:58:54.015718+0000","last_unstale":"2024-03-28T17:58:55.095113+0000","last_undegraded":"2024-03-28T17:58:55.095113+0000","last_fullsized":"2024-03-28T17:58:55.095113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:11:20.464293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.997526+0000","last_change":"2024-03-28T17:58:53.000138+0000","last_active":"2024-03-28T17:58:54.997526+0000","last_peered":"2024-03-28T17:58:54.997526+0000","last_clean":"2024-03-28T17:58:54.997526+0000","last_became_active":"2024-03-28T17:58:52.999805+0000","last_became_peered":"2024-03-28T17:58:52.999805+0000","last_unstale":"2024-03-28T17:58:54.997526+0000","last_undegraded":"2024-03-28T17:58:54.997526+0000","last_fullsized":"2024-03-28T17:58:54.997526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:13:44.327332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095084+0000","last_change":"2024-03-28T17:58:54.006123+0000","last_active":"2024-03-28T17:58:55.095084+0000","last_peered":"2024-03-28T17:58:55.095084+0000","last_clean":"2024-03-28T17:58:55.095084+0000","last_became_active":"2024-03-28T17:58:54.005714+0000","last_became_peered":"2024-03-28T17:58:54.005714+0000","last_unstale":"2024-03-28T17:58:55.095084+0000","last_undegraded":"2024-03-28T17:58:55.095084+0000","last_fullsized":"2024-03-28T17:58:55.095084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095450+0000","last_change":"2024-03-28T17:58:53.009871+0000","last_active":"2024-03-28T17:58:55.095450+0000","last_peered":"2024-03-28T17:58:55.095450+0000","last_clean":"2024-03-28T17:58:55.095450+0000","last_became_active":"2024-03-28T17:58:53.009550+0000","last_became_peered":"2024-03-28T17:58:53.009550+0000","last_unstale":"2024-03-28T17:58:55.095450+0000","last_undegraded":"2024-03-28T17:58:55.095450+0000","last_fullsized":"2024-03-28T17:58:55.095450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:03.353429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.019393+0000","last_change":"2024-03-28T17:58:54.020986+0000","last_active":"2024-03-28T17:58:55.019393+0000","last_peered":"2024-03-28T17:58:55.019393+0000","last_clean":"2024-03-28T17:58:55.019393+0000","last_became_active":"2024-03-28T17:58:54.020717+0000","last_became_peered":"2024-03-28T17:58:54.020717+0000","last_unstale":"2024-03-28T17:58:55.019393+0000","last_undegraded":"2024-03-28T17:58:55.019393+0000","last_fullsized":"2024-03-28T17:58:55.019393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:13.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:54.999845+0000","last_change":"2024-03-28T17:58:54.022957+0000","last_active":"2024-03-28T17:58:54.999845+0000","last_peered":"2024-03-28T17:58:54.999845+0000","last_clean":"2024-03-28T17:58:54.999845+0000","last_became_active":"2024-03-28T17:58:54.022234+0000","last_became_peered":"2024-03-28T17:58:54.022234+0000","last_unstale":"2024-03-28T17:58:54.999845+0000","last_undegraded":"2024-03-28T17:58:54.999845+0000","last_fullsized":"2024-03-28T17:58:54.999845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:35.685637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.097078+0000","last_change":"2024-03-28T17:58:53.010194+0000","last_active":"2024-03-28T17:58:55.097078+0000","last_peered":"2024-03-28T17:58:55.097078+0000","last_clean":"2024-03-28T17:58:55.097078+0000","last_became_active":"2024-03-28T17:58:53.009750+0000","last_became_peered":"2024-03-28T17:58:53.009750+0000","last_unstale":"2024-03-28T17:58:55.097078+0000","last_undegraded":"2024-03-28T17:58:55.097078+0000","last_fullsized":"2024-03-28T17:58:55.097078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:17:51.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.000545+0000","last_change":"2024-03-28T17:58:54.016961+0000","last_active":"2024-03-28T17:58:55.000545+0000","last_peered":"2024-03-28T17:58:55.000545+0000","last_clean":"2024-03-28T17:58:55.000545+0000","last_became_active":"2024-03-28T17:58:54.016695+0000","last_became_peered":"2024-03-28T17:58:54.016695+0000","last_unstale":"2024-03-28T17:58:55.000545+0000","last_undegraded":"2024-03-28T17:58:55.000545+0000","last_fullsized":"2024-03-28T17:58:55.000545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:28.868790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095647+0000","last_change":"2024-03-28T17:58:53.011900+0000","last_active":"2024-03-28T17:58:55.095647+0000","last_peered":"2024-03-28T17:58:55.095647+0000","last_clean":"2024-03-28T17:58:55.095647+0000","last_became_active":"2024-03-28T17:58:53.011463+0000","last_became_peered":"2024-03-28T17:58:53.011463+0000","last_unstale":"2024-03-28T17:58:55.095647+0000","last_undegraded":"2024-03-28T17:58:55.095647+0000","last_fullsized":"2024-03-28T17:58:55.095647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:15.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098095+0000","last_change":"2024-03-28T17:58:54.010789+0000","last_active":"2024-03-28T17:58:55.098095+0000","last_peered":"2024-03-28T17:58:55.098095+0000","last_clean":"2024-03-28T17:58:55.098095+0000","last_became_active":"2024-03-28T17:58:54.010520+0000","last_became_peered":"2024-03-28T17:58:54.010520+0000","last_unstale":"2024-03-28T17:58:55.098095+0000","last_undegraded":"2024-03-28T17:58:55.098095+0000","last_fullsized":"2024-03-28T17:58:55.098095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:07.934121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095653+0000","last_change":"2024-03-28T17:58:52.999999+0000","last_active":"2024-03-28T17:58:55.095653+0000","last_peered":"2024-03-28T17:58:55.095653+0000","last_clean":"2024-03-28T17:58:55.095653+0000","last_became_active":"2024-03-28T17:58:52.999659+0000","last_became_peered":"2024-03-28T17:58:52.999659+0000","last_unstale":"2024-03-28T17:58:55.095653+0000","last_undegraded":"2024-03-28T17:58:55.095653+0000","last_fullsized":"2024-03-28T17:58:55.095653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:49.218658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.095619+0000","last_change":"2024-03-28T17:58:54.010224+0000","last_active":"2024-03-28T17:58:55.095619+0000","last_peered":"2024-03-28T17:58:55.095619+0000","last_clean":"2024-03-28T17:58:55.095619+0000","last_became_active":"2024-03-28T17:58:54.008396+0000","last_became_peered":"2024-03-28T17:58:54.008396+0000","last_unstale":"2024-03-28T17:58:55.095619+0000","last_undegraded":"2024-03-28T17:58:55.095619+0000","last_fullsized":"2024-03-28T17:58:55.095619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:52.953325+0000","last_clean_scrub_stamp":"2024-03-28T17:58:52.953325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:47.114742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-28T17:58:55.098248+0000","last_change":"2024-03-28T17:58:53.008301+0000","last_active":"2024-03-28T17:58:55.098248+0000","last_peered":"2024-03-28T17:58:55.098248+0000","last_clean":"2024-03-28T17:58:55.098248+0000","last_became_active":"2024-03-28T17:58:53.007920+0000","last_became_peered":"2024-03-28T17:58:53.007920+0000","last_unstale":"2024-03-28T17:58:55.098248+0000","last_undegraded":"2024-03-28T17:58:55.098248+0000","last_fullsized":"2024-03-28T17:58:55.098248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:51.938180+0000","last_clean_scrub_stamp":"2024-03-28T17:58:51.938180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:52:20.236463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T17:59:04.655 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T17:59:04.655 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T17:59:04.655 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-28T17:59:04.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T17:59:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.909+0000 7fb08e400640 1 Processor -- start 2024-03-28T17:59:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.909+0000 7fb08e400640 1 -- start start 2024-03-28T17:59:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb08815df10 0x7fb0880b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08e400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb088161ba0 con 0x7fb0880d1080 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08e400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb088161e80 con 0x7fb0880b2ac0 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0881627b0 con 0x7fb08815df10 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08da00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb0880d1080 0x7fb0880d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53700/0 (socket says 172.21.15.62:53700) 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08da00640 1 -- 172.21.15.62:0/1841062832 learned_addr learned my addr 172.21.15.62:0/1841062832 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08d000640 1 --2- 172.21.15.62:0/1841062832 >> v2:172.21.15.62:3300/0 conn(0x7fb08815df10 0x7fb0880b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb087400640 1 -- 172.21.15.62:0/1841062832 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569105871 0 0) 0x7fb088161ba0 con 0x7fb0880d1080 2024-03-28T17:59:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb087400640 1 -- 172.21.15.62:0/1841062832 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb070003520 con 0x7fb0880d1080 2024-03-28T17:59:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08d000640 1 -- 172.21.15.62:0/1841062832 >> v1:172.21.15.81:6789/0 conn(0x7fb0880b2ac0 legacy=0x7fb0880d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08d000640 1 -- 172.21.15.62:0/1841062832 >> v1:172.21.15.62:6789/0 conn(0x7fb0880d1080 legacy=0x7fb0880d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.911+0000 7fb08d000640 1 -- 172.21.15.62:0/1841062832 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088162160 con 0x7fb08815df10 2024-03-28T17:59:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.912+0000 7fb08d000640 1 --2- 172.21.15.62:0/1841062832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb0880b9520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097a0 tx=0x7fb07c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f665c5514f39d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.912+0000 7fb087400640 1 -- 172.21.15.62:0/1841062832 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c034070 con 0x7fb08815df10 2024-03-28T17:59:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.912+0000 7fb087400640 1 -- 172.21.15.62:0/1841062832 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c030610 con 0x7fb08815df10 2024-03-28T17:59:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.912+0000 7fb087400640 1 -- 172.21.15.62:0/1841062832 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0309e0 con 0x7fb08815df10 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 -- 172.21.15.62:0/1841062832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 msgr2=0x7fb0880b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 --2- 172.21.15.62:0/1841062832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb0880b9520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097a0 tx=0x7fb07c02b960 comp rx=0 tx=0).stop 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 -- 172.21.15.62:0/1841062832 shutdown_connections 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 --2- 172.21.15.62:0/1841062832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb0880b9520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 -- 172.21.15.62:0/1841062832 >> 172.21.15.62:0/1841062832 conn(0x7fb088066fb0 msgr2=0x7fb0880673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 -- 172.21.15.62:0/1841062832 shutdown_connections 2024-03-28T17:59:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.913+0000 7fb08e400640 1 -- 172.21.15.62:0/1841062832 wait complete. 2024-03-28T17:59:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.915+0000 7fb08e400640 1 Processor -- start 2024-03-28T17:59:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.915+0000 7fb08e400640 1 -- start start 2024-03-28T17:59:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0880b2ac0 0x7fb088155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 0x7fb08815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08815e2f0 con 0x7fb08815df10 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b9a60 con 0x7fb0880d1080 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08e400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880d2590 con 0x7fb0880b2ac0 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58212/0 (socket says 172.21.15.62:58212) 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 -- 172.21.15.62:0/3814330007 learned_addr learned my addr 172.21.15.62:0/3814330007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb087e00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 0x7fb08815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08d000640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0880b2ac0 0x7fb088155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 -- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0880b2ac0 msgr2=0x7fb088155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0880b2ac0 0x7fb088155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 -- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 msgr2=0x7fb08815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 0x7fb08815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb08da00640 1 -- 172.21.15.62:0/3814330007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880b3b10 con 0x7fb08815df10 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.916+0000 7fb087e00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 0x7fb08815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb08da00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb08000cb70 tx=0x7fb08000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080017070 con 0x7fb08815df10 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08000ad40 con 0x7fb08815df10 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080012620 con 0x7fb08815df10 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880b3df0 con 0x7fb08815df10 2024-03-28T17:59:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.917+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb088156f10 con 0x7fb08815df10 2024-03-28T17:59:04.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.918+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880cf850 con 0x7fb08815df10 2024-03-28T17:59:04.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.919+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb08001a070 con 0x7fb08815df10 2024-03-28T17:59:04.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.920+0000 7fb084c00640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 0x7fb06806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:04.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.920+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb08008a840 con 0x7fb08815df10 2024-03-28T17:59:04.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.920+0000 7fb08d000640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 0x7fb06806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:04.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.921+0000 7fb08d000640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 0x7fb06806e0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb07c036000 tx=0x7fb07c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:04.924+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb0880cf850 con 0x7fb08815df10 2024-03-28T17:59:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.171+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb08804f400 con 0x7fb08815df10 2024-03-28T17:59:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.172+0000 7fb084c00640 1 -- 172.21.15.62:0/3814330007 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb0800593c0 con 0x7fb08815df10 2024-03-28T17:59:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:05.173 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 msgr2=0x7fb06806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 0x7fb06806e0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb07c036000 tx=0x7fb07c008030 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 msgr2=0x7fb088159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb08000cb70 tx=0x7fb08000a430 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 shutdown_connections 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb06806bc00 0x7fb06806e0c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08815df10 0x7fb088159260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0880d1080 0x7fb08815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 --2- 172.21.15.62:0/3814330007 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0880b2ac0 0x7fb088155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.174+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 >> 172.21.15.62:0/3814330007 conn(0x7fb088066fb0 msgr2=0x7fb0880b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.175+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 shutdown_connections 2024-03-28T17:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.175+0000 7fb08e400640 1 -- 172.21.15.62:0/3814330007 wait complete. 2024-03-28T17:59:05.193 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-28T17:59:05.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.414+0000 7fefdd400640 1 Processor -- start 2024-03-28T17:59:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.414+0000 7fefdd400640 1 -- start start 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdd400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefd8072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdd400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd8071ed0 con 0x7fefd806caa0 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdd400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd8070a40 con 0x7fefd815b9c0 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdd400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8072c50 con 0x7fefd80722b0 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fefd806caa0 0x7fefd806cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53702/0 (socket says 172.21.15.62:53702) 2024-03-28T17:59:05.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefdca00640 1 -- 172.21.15.62:0/2277843580 learned_addr learned my addr 172.21.15.62:0/2277843580 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefcf400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefd8072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834132007 0 0) 0x7fefd8071ed0 con 0x7fefd806caa0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefbc003520 con 0x7fefd806caa0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956049115 0 0) 0x7fefbc003520 con 0x7fefd806caa0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 msgr2=0x7fefd8072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefd8072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 >> v1:172.21.15.81:6789/0 conn(0x7fefd815b9c0 legacy=0x7fefd80717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80b3af0 con 0x7fefd806caa0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fefd0001670 con 0x7fefd806caa0 2024-03-28T17:59:05.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 >> v1:172.21.15.62:6789/0 conn(0x7fefd806caa0 legacy=0x7fefd806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc00ba70 con 0x7fefbc004800 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc00bc20 con 0x7fefbc0039e0 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc00be20 con 0x7fefd80722b0 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.416+0000 7fefcf400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefd7e00640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefdca00640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefcf400640 1 -- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 msgr2=0x7fefbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefcf400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd80722b0 0x7fefbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefcf400640 1 -- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefbc0039e0 msgr2=0x7fefbc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:05.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefcf400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefcf400640 1 -- 172.21.15.62:0/2277843580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc003520 con 0x7fefbc004800 2024-03-28T17:59:05.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.417+0000 7fefd7e00640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:05.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.418+0000 7fefcf400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009a80 tx=0x7fefbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0ce3b9b24273ae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.418+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0009680 con 0x7fefbc004800 2024-03-28T17:59:05.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.418+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0005990 con 0x7fefbc004800 2024-03-28T17:59:05.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.418+0000 7fefd7400640 1 -- 172.21.15.62:0/2277843580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0005dc0 con 0x7fefbc004800 2024-03-28T17:59:05.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 -- 172.21.15.62:0/2277843580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 msgr2=0x7fefbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 --2- 172.21.15.62:0/2277843580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009a80 tx=0x7fefbc005830 comp rx=0 tx=0).stop 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdca00640 1 -- 172.21.15.62:0/2277843580 reap_dead start 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 -- 172.21.15.62:0/2277843580 shutdown_connections 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 -- 172.21.15.62:0/2277843580 >> 172.21.15.62:0/2277843580 conn(0x7fefd80b1730 msgr2=0x7fefd80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 -- 172.21.15.62:0/2277843580 shutdown_connections 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.419+0000 7fefdd400640 1 -- 172.21.15.62:0/2277843580 wait complete. 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.421+0000 7fefdd400640 1 Processor -- start 2024-03-28T17:59:05.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.421+0000 7fefdd400640 1 -- start start 2024-03-28T17:59:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd814dba0 0x7fefd814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd8153790 0x7fefd80ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8070ce0 con 0x7fefbc004800 2024-03-28T17:59:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8071330 con 0x7fefd8153790 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefdd400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8070fc0 con 0x7fefd814dba0 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58238/0 (socket says 172.21.15.62:58238) 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 -- 172.21.15.62:0/1963041344 learned_addr learned my addr 172.21.15.62:0/1963041344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefcf400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd814dba0 0x7fefd814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 -- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd814dba0 msgr2=0x7fefd814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd814dba0 0x7fefd814d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 -- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd8153790 msgr2=0x7fefd80ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd8153790 0x7fefd80ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.422+0000 7fefd7e00640 1 -- 172.21.15.62:0/1963041344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8151ff0 con 0x7fefbc004800 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefd7e00640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000f80 tx=0x7fefc800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8003b80 con 0x7fefbc004800 2024-03-28T17:59:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8018070 con 0x7fefbc004800 2024-03-28T17:59:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80050d0 con 0x7fefbc004800 2024-03-28T17:59:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd8151b20 con 0x7fefbc004800 2024-03-28T17:59:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.423+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd8152570 con 0x7fefbc004800 2024-03-28T17:59:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.424+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd8073490 con 0x7fefbc004800 2024-03-28T17:59:05.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.425+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fefc8005270 con 0x7fefbc004800 2024-03-28T17:59:05.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.426+0000 7fefd4c00640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 0x7fefb406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.426+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefc808aa40 con 0x7fefbc004800 2024-03-28T17:59:05.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.427+0000 7fefcf400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 0x7fefb406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.428+0000 7fefcf400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 0x7fefb406e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefc0035000 tx=0x7fefc0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.430+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fefc80595c0 con 0x7fefbc004800 2024-03-28T17:59:05.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.636+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefd814e1a0 con 0x7fefbc004800 2024-03-28T17:59:05.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.639+0000 7fefd4c00640 1 -- 172.21.15.62:0/1963041344 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fefc8058ec0 con 0x7fefbc004800 2024-03-28T17:59:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:05.640 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:05.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 msgr2=0x7fefb406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 0x7fefb406e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefc0035000 tx=0x7fefc0030000 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 msgr2=0x7fefd814f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000f80 tx=0x7fefc800ef80 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 shutdown_connections 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fefb406bbd0 0x7fefb406e090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd8153790 0x7fefd80ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fefd814dba0 0x7fefd814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 --2- 172.21.15.62:0/1963041344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc004800 0x7fefd814f920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.642+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 >> 172.21.15.62:0/1963041344 conn(0x7fefd80b1730 msgr2=0x7fefd80cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.643+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 shutdown_connections 2024-03-28T17:59:05.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.643+0000 7fefdd400640 1 -- 172.21.15.62:0/1963041344 wait complete. 2024-03-28T17:59:05.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:05.662 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.877+0000 7f95d4800640 1 Processor -- start 2024-03-28T17:59:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.878+0000 7f95d4800640 1 -- start start 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.879+0000 7f95d4800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f95cc071dd0 0x7f95cc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.879+0000 7f95d4800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95cc0b6fb0 0x7f95cc0d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.879+0000 7f95d4800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc06cdd0 con 0x7f95cc15e020 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.879+0000 7f95d4800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0d1af0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.879+0000 7f95d4800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0d1dd0 con 0x7f95cc071dd0 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d1e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f95cc15e020 0x7f95cc0716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53714/0 (socket says 172.21.15.62:53714) 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d1e00640 1 -- 172.21.15.62:0/262467525 learned_addr learned my addr 172.21.15.62:0/262467525 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d2800640 1 --2- 172.21.15.62:0/262467525 >> v2:172.21.15.62:3300/0 conn(0x7f95cc0b6fb0 0x7f95cc0d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d1400640 1 --2- 172.21.15.62:0/262467525 >> v2:172.21.15.81:3300/0 conn(0x7f95cc071dd0 0x7f95cc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d0a00640 1 -- 172.21.15.62:0/262467525 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048663272 0 0) 0x7f95cc06cdd0 con 0x7f95cc15e020 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d0a00640 1 -- 172.21.15.62:0/262467525 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b4003520 con 0x7f95cc15e020 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d2800640 1 -- 172.21.15.62:0/262467525 >> v2:172.21.15.81:3300/0 conn(0x7f95cc071dd0 msgr2=0x7f95cc0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d2800640 1 --2- 172.21.15.62:0/262467525 >> v2:172.21.15.81:3300/0 conn(0x7f95cc071dd0 0x7f95cc0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d2800640 1 -- 172.21.15.62:0/262467525 >> v1:172.21.15.62:6789/0 conn(0x7f95cc15e020 legacy=0x7f95cc0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.880+0000 7f95d2800640 1 -- 172.21.15.62:0/262467525 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0d20b0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.881+0000 7f95d2800640 1 --2- 172.21.15.62:0/262467525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc0d30d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0d01b0 tx=0x7f95c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1d47de1a4020fc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d0a00640 1 -- 172.21.15.62:0/262467525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8039070 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d0a00640 1 -- 172.21.15.62:0/262467525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c802cc80 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d0a00640 1 -- 172.21.15.62:0/262467525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8034660 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 -- 172.21.15.62:0/262467525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 msgr2=0x7f95cc0d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 --2- 172.21.15.62:0/262467525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc0d30d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0d01b0 tx=0x7f95c802bed0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 -- 172.21.15.62:0/262467525 shutdown_connections 2024-03-28T17:59:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 --2- 172.21.15.62:0/262467525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc0d30d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 --2- 172.21.15.62:0/262467525 >> v2:172.21.15.81:3300/0 conn(0x7f95cc071dd0 0x7f95cc0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.882+0000 7f95d4800640 1 -- 172.21.15.62:0/262467525 >> 172.21.15.62:0/262467525 conn(0x7f95cc0d3d80 msgr2=0x7f95cc0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.883+0000 7f95d4800640 1 -- 172.21.15.62:0/262467525 shutdown_connections 2024-03-28T17:59:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.883+0000 7f95d4800640 1 -- 172.21.15.62:0/262467525 wait complete. 2024-03-28T17:59:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.884+0000 7f95d4800640 1 Processor -- start 2024-03-28T17:59:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.884+0000 7f95d4800640 1 -- start start 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95cc071dd0 0x7f95cc0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 0x7f95cc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15e400 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0d2670 con 0x7f95cc15e020 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d4800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15e580 con 0x7f95cc071dd0 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.885+0000 7f95d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58254/0 (socket says 172.21.15.62:58254) 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 -- 172.21.15.62:0/4279741732 learned_addr learned my addr 172.21.15.62:0/4279741732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1e00640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95cc071dd0 0x7f95cc0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d2800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 0x7f95cc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 -- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95cc071dd0 msgr2=0x7f95cc0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95cc071dd0 0x7f95cc0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 -- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 msgr2=0x7f95cc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 0x7f95cc15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 -- 172.21.15.62:0/4279741732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0b4dc0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d2800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 0x7f95cc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.886+0000 7f95d1400640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9b0 tx=0x7f95bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.887+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.887+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00c9a0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.887+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00cd90 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.887+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc0b50a0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.887+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc0b5660 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.888+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc0680c0 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.889+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f95bc01a070 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.890+0000 7f95c6000640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 0x7f95b006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.890+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95bc08ae50 con 0x7f95cc0b6fb0 2024-03-28T17:59:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.891+0000 7f95d1e00640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 0x7f95b006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:05.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.892+0000 7f95d1e00640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 0x7f95b006e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95c0007fd0 tx=0x7f95c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:05.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:05.894+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f95bc059940 con 0x7f95cc0b6fb0 2024-03-28T17:59:06.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.091+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95cc0cebd0 con 0x7f95cc0b6fb0 2024-03-28T17:59:06.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.092+0000 7f95c6000640 1 -- 172.21.15.62:0/4279741732 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95bc059240 con 0x7f95cc0b6fb0 2024-03-28T17:59:06.092 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:06.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 msgr2=0x7f95b006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 0x7f95b006e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95c0007fd0 tx=0x7f95c0009040 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 msgr2=0x7f95cc1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9b0 tx=0x7f95bc00ee70 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 shutdown_connections 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f95b006bbd0 0x7f95b006e090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95cc15e020 0x7f95cc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95cc0b6fb0 0x7f95cc1f4b30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 --2- 172.21.15.62:0/4279741732 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f95cc071dd0 0x7f95cc0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 >> 172.21.15.62:0/4279741732 conn(0x7f95cc0d3d80 msgr2=0x7f95cc0716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 shutdown_connections 2024-03-28T17:59:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.095+0000 7f95d4800640 1 -- 172.21.15.62:0/4279741732 wait complete. 2024-03-28T17:59:06.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:06.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.323+0000 7f7779e00640 1 Processor -- start 2024-03-28T17:59:06.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.323+0000 7f7779e00640 1 -- start start 2024-03-28T17:59:06.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7779e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f77740ae720 0x7f77740aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7779e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7774072690 0x7f777415f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7779e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777415d1b0 con 0x7f7774072b00 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7779e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740af040 con 0x7f77740ae720 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7779e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7774070d50 con 0x7f7774072690 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.325+0000 7f7772a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f77740ae720 0x7f77740aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f77740ae720 0x7f77740aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58268/0 (socket says 172.21.15.62:58268) 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 -- 172.21.15.62:0/2781808891 learned_addr learned my addr 172.21.15.62:0/2781808891 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7773400640 1 --1- 172.21.15.62:0/2781808891 >> v1:172.21.15.62:6789/0 conn(0x7f7774072b00 0x7f777415fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53726/0 (socket says 172.21.15.62:53726) 2024-03-28T17:59:06.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772000640 1 --2- 172.21.15.62:0/2781808891 >> v2:172.21.15.81:3300/0 conn(0x7f7774072690 0x7f777415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 -- 172.21.15.62:0/2781808891 >> v2:172.21.15.81:3300/0 conn(0x7f7774072690 msgr2=0x7f777415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 --2- 172.21.15.62:0/2781808891 >> v2:172.21.15.81:3300/0 conn(0x7f7774072690 0x7f777415f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 -- 172.21.15.62:0/2781808891 >> v1:172.21.15.62:6789/0 conn(0x7f7774072b00 legacy=0x7f777415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.326+0000 7f7772a00640 1 -- 172.21.15.62:0/2781808891 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774071030 con 0x7f77740ae720 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.327+0000 7f7772a00640 1 --2- 172.21.15.62:0/2781808891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77740ae720 0x7f77740aeb00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7774073370 tx=0x7f776402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6189a21da04f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7771600640 1 -- 172.21.15.62:0/2781808891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764034070 con 0x7f77740ae720 2024-03-28T17:59:06.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7771600640 1 -- 172.21.15.62:0/2781808891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764030860 con 0x7f77740ae720 2024-03-28T17:59:06.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7771600640 1 -- 172.21.15.62:0/2781808891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764030c30 con 0x7f77740ae720 2024-03-28T17:59:06.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 -- 172.21.15.62:0/2781808891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77740ae720 msgr2=0x7f77740aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 --2- 172.21.15.62:0/2781808891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77740ae720 0x7f77740aeb00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7774073370 tx=0x7f776402b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 -- 172.21.15.62:0/2781808891 shutdown_connections 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 --2- 172.21.15.62:0/2781808891 >> v2:172.21.15.81:3300/0 conn(0x7f7774072690 0x7f777415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 --2- 172.21.15.62:0/2781808891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77740ae720 0x7f77740aeb00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 -- 172.21.15.62:0/2781808891 >> 172.21.15.62:0/2781808891 conn(0x7f77740b1650 msgr2=0x7f77740b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 -- 172.21.15.62:0/2781808891 shutdown_connections 2024-03-28T17:59:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.328+0000 7f7779e00640 1 -- 172.21.15.62:0/2781808891 wait complete. 2024-03-28T17:59:06.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.330+0000 7f7779e00640 1 Processor -- start 2024-03-28T17:59:06.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.330+0000 7f7779e00640 1 -- start start 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 0x7f7774151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 0x7f77741f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740718d0 con 0x7f7774072690 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740715f0 con 0x7f7774072b00 2024-03-28T17:59:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7779e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7774070a40 con 0x7f77740ae720 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7772a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7772a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58272/0 (socket says 172.21.15.62:58272) 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.331+0000 7f7772a00640 1 -- 172.21.15.62:0/2367057626 learned_addr learned my addr 172.21.15.62:0/2367057626 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772000640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 0x7f7774151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7773400640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 0x7f77741f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 -- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 msgr2=0x7f77741f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 0x7f77741f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 -- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 msgr2=0x7f7774151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 0x7f7774151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 -- 172.21.15.62:0/2367057626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77741f3830 con 0x7f7774072690 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772000640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 0x7f7774151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7773400640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 0x7f77741f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.332+0000 7f7772a00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f776402b9e0 tx=0x7f77640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.333+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764034040 con 0x7f7774072690 2024-03-28T17:59:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.333+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764039d90 con 0x7f7774072690 2024-03-28T17:59:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.333+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764043610 con 0x7f7774072690 2024-03-28T17:59:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.333+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741f3ad0 con 0x7f7774072690 2024-03-28T17:59:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.333+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741f4090 con 0x7f7774072690 2024-03-28T17:59:06.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.334+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77740680c0 con 0x7f7774072690 2024-03-28T17:59:06.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.335+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f776404a060 con 0x7f7774072690 2024-03-28T17:59:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.336+0000 7f776b400640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 0x7f775006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.336+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77640acac0 con 0x7f7774072690 2024-03-28T17:59:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.337+0000 7f7772000640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 0x7f775006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.338+0000 7f7772000640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 0x7f775006e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7774151ae0 tx=0x7f775c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.341+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f776403e070 con 0x7f7774072690 2024-03-28T17:59:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.542+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77740b2420 con 0x7f7774072690 2024-03-28T17:59:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.544+0000 7f776b400640 1 -- 172.21.15.62:0/2367057626 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f776407b640 con 0x7f7774072690 2024-03-28T17:59:06.544 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.546+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 msgr2=0x7f775006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.546+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 0x7f775006e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7774151ae0 tx=0x7f775c008040 comp rx=0 tx=0).stop 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.546+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 msgr2=0x7f777413e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.546+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f776402b9e0 tx=0x7f77640046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 shutdown_connections 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f775006bbd0 0x7f775006e090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77740ae720 0x7f77741f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7774072b00 0x7f7774151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 --2- 172.21.15.62:0/2367057626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7774072690 0x7f777413e170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 >> 172.21.15.62:0/2367057626 conn(0x7f77740b1650 msgr2=0x7f77740b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:06.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 shutdown_connections 2024-03-28T17:59:06.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.547+0000 7f7779e00640 1 -- 172.21.15.62:0/2367057626 wait complete. 2024-03-28T17:59:06.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:06.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.785+0000 7f43e8c00640 1 Processor -- start 2024-03-28T17:59:06.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.785+0000 7f43e8c00640 1 -- start start 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e8c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f43e40ae720 0x7f43e40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e8c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43e4072b00 0x7f43e415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e8c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e415d1b0 con 0x7f43e4072690 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e8c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e40af040 con 0x7f43e40ae720 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e8c00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e4070d50 con 0x7f43e4072b00 2024-03-28T17:59:06.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f43e40ae720 0x7f43e40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f43e40ae720 0x7f43e40aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58286/0 (socket says 172.21.15.62:58286) 2024-03-28T17:59:06.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e1e00640 1 -- 172.21.15.62:0/2727186602 learned_addr learned my addr 172.21.15.62:0/2727186602 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:06.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.787+0000 7f43e2800640 1 --2- 172.21.15.62:0/2727186602 >> v2:172.21.15.81:3300/0 conn(0x7f43e4072b00 0x7f43e415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.788+0000 7f43e1e00640 1 -- 172.21.15.62:0/2727186602 >> v2:172.21.15.81:3300/0 conn(0x7f43e4072b00 msgr2=0x7f43e415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:06.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.788+0000 7f43e1e00640 1 --2- 172.21.15.62:0/2727186602 >> v2:172.21.15.81:3300/0 conn(0x7f43e4072b00 0x7f43e415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.788+0000 7f43e1e00640 1 -- 172.21.15.62:0/2727186602 >> v1:172.21.15.81:6789/0 conn(0x7f43e4072690 legacy=0x7f43e415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.788+0000 7f43e1e00640 1 -- 172.21.15.62:0/2727186602 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4071030 con 0x7f43e40ae720 2024-03-28T17:59:06.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.789+0000 7f43e1e00640 1 --2- 172.21.15.62:0/2727186602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e40ae720 0x7f43e40aeb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43e406eab0 tx=0x7f43d402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ad0d9fd0004238 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.790+0000 7f43e0a00640 1 -- 172.21.15.62:0/2727186602 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d402cb20 con 0x7f43e40ae720 2024-03-28T17:59:06.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.790+0000 7f43e0a00640 1 -- 172.21.15.62:0/2727186602 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d402ccc0 con 0x7f43e40ae720 2024-03-28T17:59:06.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.790+0000 7f43e0a00640 1 -- 172.21.15.62:0/2727186602 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4034610 con 0x7f43e40ae720 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 -- 172.21.15.62:0/2727186602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e40ae720 msgr2=0x7f43e40aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 --2- 172.21.15.62:0/2727186602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e40ae720 0x7f43e40aeb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43e406eab0 tx=0x7f43d402bf80 comp rx=0 tx=0).stop 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 -- 172.21.15.62:0/2727186602 shutdown_connections 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 --2- 172.21.15.62:0/2727186602 >> v2:172.21.15.81:3300/0 conn(0x7f43e4072b00 0x7f43e415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 --2- 172.21.15.62:0/2727186602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e40ae720 0x7f43e40aeb00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 -- 172.21.15.62:0/2727186602 >> 172.21.15.62:0/2727186602 conn(0x7f43e40b1650 msgr2=0x7f43e40b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 -- 172.21.15.62:0/2727186602 shutdown_connections 2024-03-28T17:59:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.791+0000 7f43e8c00640 1 -- 172.21.15.62:0/2727186602 wait complete. 2024-03-28T17:59:06.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.793+0000 7f43e8c00640 1 Processor -- start 2024-03-28T17:59:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.793+0000 7f43e8c00640 1 -- start start 2024-03-28T17:59:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 0x7f43e4142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 0x7f43e41e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40718d0 con 0x7f43e4072b00 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40715f0 con 0x7f43e4072690 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e8c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e4070a40 con 0x7f43e40ae720 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58302/0 (socket says 172.21.15.62:58302) 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e1400640 1 -- 172.21.15.62:0/1147045377 learned_addr learned my addr 172.21.15.62:0/1147045377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e1e00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 0x7f43e4142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.794+0000 7f43e2800640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 0x7f43e41e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 -- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 msgr2=0x7f43e41e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 0x7f43e41e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 -- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 msgr2=0x7f43e4142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 0x7f43e4142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 -- 172.21.15.62:0/1147045377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e41428b0 con 0x7f43e4072b00 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1e00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 0x7f43e4142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e2800640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 0x7f43e41e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e1400640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c400 tx=0x7f43d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00052d0 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0005490 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e41549f0 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0007260 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.795+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e4154fb0 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.797+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e40680c0 con 0x7f43e4072b00 2024-03-28T17:59:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.798+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f43d0007400 con 0x7f43e4072b00 2024-03-28T17:59:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.799+0000 7f43da000640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 0x7f43c406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.799+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43d0089ff0 con 0x7f43e4072b00 2024-03-28T17:59:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.799+0000 7f43e1e00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 0x7f43c406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:06.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.800+0000 7f43e1e00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 0x7f43c406e090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43d4035000 tx=0x7f43d4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:06.803+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f43e41428b0 con 0x7f43e4072b00 2024-03-28T17:59:07.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.005+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43e40b2420 con 0x7f43e4072b00 2024-03-28T17:59:07.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.006+0000 7f43da000640 1 -- 172.21.15.62:0/1147045377 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43d0058ad0 con 0x7f43e4072b00 2024-03-28T17:59:07.006 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.008+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 msgr2=0x7f43c406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.008+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 0x7f43c406e090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43d4035000 tx=0x7f43d4030000 comp rx=0 tx=0).stop 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 msgr2=0x7f43e4155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c400 tx=0x7f43d000c8c0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 shutdown_connections 2024-03-28T17:59:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43c406bbd0 0x7f43c406e090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43e40ae720 0x7f43e41e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4072b00 0x7f43e4155a90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 --2- 172.21.15.62:0/1147045377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43e4072690 0x7f43e4142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 >> 172.21.15.62:0/1147045377 conn(0x7f43e40b1650 msgr2=0x7f43e415c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 shutdown_connections 2024-03-28T17:59:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.009+0000 7f43e8c00640 1 -- 172.21.15.62:0/1147045377 wait complete. 2024-03-28T17:59:07.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T17:59:07.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.248+0000 7f7172800640 1 Processor -- start 2024-03-28T17:59:07.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.248+0000 7f7172800640 1 -- start start 2024-03-28T17:59:07.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.250+0000 7f7172800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f716c15dd60 0x7f716c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.250+0000 7f7172800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c0b6e60 con 0x7f716c0b5e00 2024-03-28T17:59:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.250+0000 7f7172800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c0b7100 con 0x7f716c0b99f0 2024-03-28T17:59:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.250+0000 7f7172800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0b6c30 con 0x7f716c15dd60 2024-03-28T17:59:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f7171e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f716c0b5e00 0x7f716c0b7ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53742/0 (socket says 172.21.15.62:53742) 2024-03-28T17:59:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f7171e00640 1 -- 172.21.15.62:0/2138395451 learned_addr learned my addr 172.21.15.62:0/2138395451 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f7171400640 1 --2- 172.21.15.62:0/2138395451 >> v2:172.21.15.81:3300/0 conn(0x7f716c15dd60 0x7f716c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311372563 0 0) 0x7f716c0b6e60 con 0x7f716c0b5e00 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7154003520 con 0x7f716c0b5e00 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3959358048 0 0) 0x7f7154003520 con 0x7f716c0b5e00 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 >> v2:172.21.15.81:3300/0 conn(0x7f716c15dd60 msgr2=0x7f716c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 --2- 172.21.15.62:0/2138395451 >> v2:172.21.15.81:3300/0 conn(0x7f716c15dd60 0x7f716c0b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.251+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 >> v1:172.21.15.81:6789/0 conn(0x7f716c0b99f0 legacy=0x7f716c0b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0b73e0 con 0x7f716c0b5e00 2024-03-28T17:59:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f7164001a10 con 0x7f716c0b5e00 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 >> v1:172.21.15.62:6789/0 conn(0x7f716c0b5e00 legacy=0x7f716c0b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 0x7f7154003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71540051c0 0x7f715400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715400c070 con 0x7f7154003a00 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715400c220 con 0x7f71540051c0 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715400c420 con 0x7f7154004800 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 0x7f7154003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171e00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71540051c0 0x7f715400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7170a00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 -- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7154004800 msgr2=0x7f7154004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 -- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71540051c0 msgr2=0x7f715400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71540051c0 0x7f715400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171400640 1 -- 172.21.15.62:0/2138395451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154003520 con 0x7f7154003a00 2024-03-28T17:59:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.252+0000 7f7171e00640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71540051c0 0x7f715400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.253+0000 7f7171e00640 1 -- 172.21.15.62:0/2138395451 reap_dead start 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.254+0000 7f7171400640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 0x7f7154003e20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7160009ac0 tx=0x7f7154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a9990ec259422f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.254+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160039070 con 0x7f7154003a00 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.254+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716002c6b0 con 0x7f7154003a00 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.254+0000 7f716be00640 1 -- 172.21.15.62:0/2138395451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160009690 con 0x7f7154003a00 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 -- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 msgr2=0x7f7154003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 0x7f7154003e20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7160009ac0 tx=0x7f7154005830 comp rx=0 tx=0).stop 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 -- 172.21.15.62:0/2138395451 shutdown_connections 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 --2- 172.21.15.62:0/2138395451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7154003a00 0x7f7154003e20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 -- 172.21.15.62:0/2138395451 >> 172.21.15.62:0/2138395451 conn(0x7f716c071ae0 msgr2=0x7f716c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 -- 172.21.15.62:0/2138395451 shutdown_connections 2024-03-28T17:59:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.255+0000 7f7172800640 1 -- 172.21.15.62:0/2138395451 wait complete. 2024-03-28T17:59:07.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.257+0000 7f7172800640 1 Processor -- start 2024-03-28T17:59:07.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.257+0000 7f7172800640 1 -- start start 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7154003a00 0x7f716c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f716c0c77a0 0x7f716c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c15e140 con 0x7f716c0d2fc0 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b79a0 con 0x7f7154003a00 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7172800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b6370 con 0x7f716c0c77a0 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58312/0 (socket says 172.21.15.62:58312) 2024-03-28T17:59:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 -- 172.21.15.62:0/476822428 learned_addr learned my addr 172.21.15.62:0/476822428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171400640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7154003a00 0x7f716c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7170a00640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f716c0c77a0 0x7f716c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 -- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f716c0c77a0 msgr2=0x7f716c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f716c0c77a0 0x7f716c0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 -- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7154003a00 msgr2=0x7f716c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7154003a00 0x7f716c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.258+0000 7f7171e00640 1 -- 172.21.15.62:0/476822428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0cbf60 con 0x7f716c0d2fc0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7171e00640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f716400db80 tx=0x7f716400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164004d60 con 0x7f716c0d2fc0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400bb50 con 0x7f716c0d2fc0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164007070 con 0x7f716c0d2fc0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c0c7ca0 con 0x7f716c0d2fc0 2024-03-28T17:59:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.259+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c0c8260 con 0x7f716c0d2fc0 2024-03-28T17:59:07.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.260+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c072b90 con 0x7f716c0d2fc0 2024-03-28T17:59:07.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.261+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f716401b020 con 0x7f716c0d2fc0 2024-03-28T17:59:07.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.262+0000 7f7169600640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 0x7f714c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.262+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f716408ab50 con 0x7f716c0d2fc0 2024-03-28T17:59:07.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.263+0000 7f7171400640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 0x7f714c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.264+0000 7f7171400640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 0x7f714c06e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7160036000 tx=0x7f7160030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.266+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f71640595a0 con 0x7f716c0d2fc0 2024-03-28T17:59:07.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.505+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f716c04f4e0 con 0x7f716c0d2fc0 2024-03-28T17:59:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.507+0000 7f7169600640 1 -- 172.21.15.62:0/476822428 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f7164058ea0 con 0x7f716c0d2fc0 2024-03-28T17:59:07.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:07.507 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T17:59:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T17:59:07.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 msgr2=0x7f714c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 0x7f714c06e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7160036000 tx=0x7f7160030000 comp rx=0 tx=0).stop 2024-03-28T17:59:07.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 msgr2=0x7f716c0d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f716400db80 tx=0x7f716400b4a0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 shutdown_connections 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f714c06bbd0 0x7f714c06e090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f716c0d2fc0 0x7f716c0d3440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f716c0c77a0 0x7f716c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 --2- 172.21.15.62:0/476822428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7154003a00 0x7f716c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 >> 172.21.15.62:0/476822428 conn(0x7f716c071ae0 msgr2=0x7f716c15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 shutdown_connections 2024-03-28T17:59:07.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.509+0000 7f7172800640 1 -- 172.21.15.62:0/476822428 wait complete. 2024-03-28T17:59:07.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:07.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.744+0000 7fcdcea00640 1 Processor -- start 2024-03-28T17:59:07.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.745+0000 7fcdcea00640 1 -- start start 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdcea00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcdc8160090 0x7fcdc8070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdcea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc8072ab0 0x7fcdc8070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdcea00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80cebb0 con 0x7fcdc81456d0 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdcea00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80cee90 con 0x7fcdc8072ab0 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdcea00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80cf170 con 0x7fcdc8160090 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdce000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc8072ab0 0x7fcdc8070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdce000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc8072ab0 0x7fcdc8070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58322/0 (socket says 172.21.15.62:58322) 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.746+0000 7fcdce000640 1 -- 172.21.15.62:0/2779747554 learned_addr learned my addr 172.21.15.62:0/2779747554 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.747+0000 7fcdccc00640 1 --2- 172.21.15.62:0/2779747554 >> v2:172.21.15.81:3300/0 conn(0x7fcdc8160090 0x7fcdc8070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.747+0000 7fcdce000640 1 -- 172.21.15.62:0/2779747554 >> v2:172.21.15.81:3300/0 conn(0x7fcdc8160090 msgr2=0x7fcdc8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.747+0000 7fcdce000640 1 --2- 172.21.15.62:0/2779747554 >> v2:172.21.15.81:3300/0 conn(0x7fcdc8160090 0x7fcdc8070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.747+0000 7fcdce000640 1 -- 172.21.15.62:0/2779747554 >> v1:172.21.15.81:6789/0 conn(0x7fcdc81456d0 legacy=0x7fcdc815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.747+0000 7fcdce000640 1 -- 172.21.15.62:0/2779747554 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80cf450 con 0x7fcdc8072ab0 2024-03-28T17:59:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.748+0000 7fcdce000640 1 --2- 172.21.15.62:0/2779747554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8072ab0 0x7fcdc8070f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80724a0 tx=0x7fcdc002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb87de915c4fc3d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.748+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2779747554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002c9f0 con 0x7fcdc8072ab0 2024-03-28T17:59:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.748+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2779747554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc002cb90 con 0x7fcdc8072ab0 2024-03-28T17:59:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.748+0000 7fcdc7e00640 1 -- 172.21.15.62:0/2779747554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00343f0 con 0x7fcdc8072ab0 2024-03-28T17:59:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 -- 172.21.15.62:0/2779747554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8072ab0 msgr2=0x7fcdc8070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2779747554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8072ab0 0x7fcdc8070f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80724a0 tx=0x7fcdc002be50 comp rx=0 tx=0).stop 2024-03-28T17:59:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 -- 172.21.15.62:0/2779747554 shutdown_connections 2024-03-28T17:59:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2779747554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8072ab0 0x7fcdc8070f80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 --2- 172.21.15.62:0/2779747554 >> v2:172.21.15.81:3300/0 conn(0x7fcdc8160090 0x7fcdc8070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 -- 172.21.15.62:0/2779747554 >> 172.21.15.62:0/2779747554 conn(0x7fcdc80b1730 msgr2=0x7fcdc80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 -- 172.21.15.62:0/2779747554 shutdown_connections 2024-03-28T17:59:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.749+0000 7fcdcea00640 1 -- 172.21.15.62:0/2779747554 wait complete. 2024-03-28T17:59:07.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.751+0000 7fcdcea00640 1 Processor -- start 2024-03-28T17:59:07.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.751+0000 7fcdcea00640 1 -- start start 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc8072ab0 0x7fcdc8142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 0x7fcdc81f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc806dd90 con 0x7fcdc8160090 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cf730 con 0x7fcdc81456d0 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcea00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc806df10 con 0x7fcdc8072ab0 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdce000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdce000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58330/0 (socket says 172.21.15.62:58330) 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdce000640 1 -- 172.21.15.62:0/3971485358 learned_addr learned my addr 172.21.15.62:0/3971485358 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdcd600640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc8072ab0 0x7fcdc8142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.752+0000 7fcdccc00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 0x7fcdc81f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 -- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc8072ab0 msgr2=0x7fcdc8142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc8072ab0 0x7fcdc8142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 -- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 msgr2=0x7fcdc81f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 0x7fcdc81f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 -- 172.21.15.62:0/3971485358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc81434f0 con 0x7fcdc8160090 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdccc00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 0x7fcdc81f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:07.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdce000640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c460 tx=0x7fcdc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0043020 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc003e450 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81437d0 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0009510 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.753+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc8143d90 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.754+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc8073490 con 0x7fcdc8160090 2024-03-28T17:59:07.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.755+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcdc0046020 con 0x7fcdc8160090 2024-03-28T17:59:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.756+0000 7fcdc5600640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 0x7fcda806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.756+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdc00ad270 con 0x7fcdc8160090 2024-03-28T17:59:07.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.757+0000 7fcdcd600640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 0x7fcda806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:07.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.758+0000 7fcdcd600640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 0x7fcda806e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc004ea0 tx=0x7fcdbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.761+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fcdc004d030 con 0x7fcdc8160090 2024-03-28T17:59:07.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.966+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdc80b1510 con 0x7fcdc8160090 2024-03-28T17:59:07.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.969+0000 7fcdc5600640 1 -- 172.21.15.62:0/3971485358 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fcdc007bd50 con 0x7fcdc8160090 2024-03-28T17:59:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:07.970 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:07.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 msgr2=0x7fcda806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 0x7fcda806e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc004ea0 tx=0x7fcdbc009040 comp rx=0 tx=0).stop 2024-03-28T17:59:07.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 msgr2=0x7fcdc815d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:07.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c460 tx=0x7fcdc0004a40 comp rx=0 tx=0).stop 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 shutdown_connections 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fcda806bbd0 0x7fcda806e090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc8160090 0x7fcdc815d2e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdc81456d0 0x7fcdc81f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 --2- 172.21.15.62:0/3971485358 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fcdc8072ab0 0x7fcdc8142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.972+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 >> 172.21.15.62:0/3971485358 conn(0x7fcdc80b1730 msgr2=0x7fcdc815b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.973+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 shutdown_connections 2024-03-28T17:59:07.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:07.973+0000 7fcdcea00640 1 -- 172.21.15.62:0/3971485358 wait complete. 2024-03-28T17:59:07.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:07.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:08.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.211+0000 7fa952200640 1 Processor -- start 2024-03-28T17:59:08.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.211+0000 7fa952200640 1 -- start start 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.212+0000 7fa952200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.212+0000 7fa952200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa952200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c0b9760 con 0x7fa94c0ce910 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa952200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c1627f0 con 0x7fa94c0b6cb0 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa952200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b7240 con 0x7fa94c15dd10 2024-03-28T17:59:08.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa94be00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa950e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa94be00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48452/0 (socket says 172.21.15.62:48452) 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa94be00640 1 -- 172.21.15.62:0/1820227241 learned_addr learned my addr 172.21.15.62:0/1820227241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.213+0000 7fa950e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:51960/0 (socket says 172.21.15.62:51960) 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.214+0000 7fa950e00640 1 -- 172.21.15.62:0/1820227241 >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 msgr2=0x7fa94c1622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.214+0000 7fa950e00640 1 --2- 172.21.15.62:0/1820227241 >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.214+0000 7fa950e00640 1 -- 172.21.15.62:0/1820227241 >> v1:172.21.15.81:6789/0 conn(0x7fa94c0ce910 legacy=0x7fa94c0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.214+0000 7fa950e00640 1 -- 172.21.15.62:0/1820227241 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0b7520 con 0x7fa94c15dd10 2024-03-28T17:59:08.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.214+0000 7fa94be00640 1 --2- 172.21.15.62:0/1820227241 >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:08.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.215+0000 7fa950e00640 1 --2- 172.21.15.62:0/1820227241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009d90 tx=0x7fa93c02bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca23eec6d1cfaba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa94b400640 1 -- 172.21.15.62:0/1820227241 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c039070 con 0x7fa94c15dd10 2024-03-28T17:59:08.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa94b400640 1 -- 172.21.15.62:0/1820227241 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c02ccf0 con 0x7fa94c15dd10 2024-03-28T17:59:08.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa94b400640 1 -- 172.21.15.62:0/1820227241 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c034660 con 0x7fa94c15dd10 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 -- 172.21.15.62:0/1820227241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 msgr2=0x7fa94c0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 --2- 172.21.15.62:0/1820227241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009d90 tx=0x7fa93c02bf40 comp rx=0 tx=0).stop 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 -- 172.21.15.62:0/1820227241 shutdown_connections 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 --2- 172.21.15.62:0/1820227241 >> v2:172.21.15.81:3300/0 conn(0x7fa94c0b6cb0 0x7fa94c1622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 --2- 172.21.15.62:0/1820227241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c15dd10 0x7fa94c0d1c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.216+0000 7fa952200640 1 -- 172.21.15.62:0/1820227241 >> 172.21.15.62:0/1820227241 conn(0x7fa94c071ae0 msgr2=0x7fa94c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.217+0000 7fa952200640 1 -- 172.21.15.62:0/1820227241 shutdown_connections 2024-03-28T17:59:08.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.217+0000 7fa952200640 1 -- 172.21.15.62:0/1820227241 wait complete. 2024-03-28T17:59:08.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.218+0000 7fa952200640 1 Processor -- start 2024-03-28T17:59:08.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.218+0000 7fa952200640 1 -- start start 2024-03-28T17:59:08.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa94c0b6cb0 0x7fa94c0d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c0ce910 0x7fa94c0d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b7dc0 con 0x7fa94c15dd10 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b7ae0 con 0x7fa94c0b6cb0 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa952200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c15e0f0 con 0x7fa94c0ce910 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.219+0000 7fa951800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58344/0 (socket says 172.21.15.62:58344) 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 -- 172.21.15.62:0/3419487285 learned_addr learned my addr 172.21.15.62:0/3419487285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa94be00640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c0ce910 0x7fa94c0d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa950e00640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa94c0b6cb0 0x7fa94c0d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 -- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c0ce910 msgr2=0x7fa94c0d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c0ce910 0x7fa94c0d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 -- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa94c0b6cb0 msgr2=0x7fa94c0d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa94c0b6cb0 0x7fa94c0d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 -- 172.21.15.62:0/3419487285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0cd500 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa951800640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa944001eb0 tx=0x7fa94400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017070 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9440052d0 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.220+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944003810 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.221+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c0cd7e0 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.221+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c15e360 con 0x7fa94c15dd10 2024-03-28T17:59:08.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.222+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c04f400 con 0x7fa94c15dd10 2024-03-28T17:59:08.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.223+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa94401a070 con 0x7fa94c15dd10 2024-03-28T17:59:08.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.224+0000 7fa948c00640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 0x7fa92806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.224+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa944089e30 con 0x7fa94c15dd10 2024-03-28T17:59:08.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.224+0000 7fa950e00640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 0x7fa92806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.225+0000 7fa950e00640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 0x7fa92806e090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa93c036000 tx=0x7fa93c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.228+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa944058910 con 0x7fa94c15dd10 2024-03-28T17:59:08.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.424+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa94c0d1830 con 0x7fa94c15dd10 2024-03-28T17:59:08.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.425+0000 7fa948c00640 1 -- 172.21.15.62:0/3419487285 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa944058210 con 0x7fa94c15dd10 2024-03-28T17:59:08.425 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.427+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 msgr2=0x7fa92806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.427+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 0x7fa92806e090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa93c036000 tx=0x7fa93c030000 comp rx=0 tx=0).stop 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.427+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 msgr2=0x7fa94c1605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.427+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa944001eb0 tx=0x7fa94400c890 comp rx=0 tx=0).stop 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 shutdown_connections 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa92806bbd0 0x7fa92806e090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94c15dd10 0x7fa94c1605d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa94c0ce910 0x7fa94c0d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 --2- 172.21.15.62:0/3419487285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa94c0b6cb0 0x7fa94c0d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 >> 172.21.15.62:0/3419487285 conn(0x7fa94c071ae0 msgr2=0x7fa94c1590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:08.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 shutdown_connections 2024-03-28T17:59:08.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.428+0000 7fa952200640 1 -- 172.21.15.62:0/3419487285 wait complete. 2024-03-28T17:59:08.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:08.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.668+0000 7f80ef000640 1 Processor -- start 2024-03-28T17:59:08.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.668+0000 7f80ef000640 1 -- start start 2024-03-28T17:59:08.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80ef000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80e81617a0 0x7f80e8071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80ef000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e80ce7f0 con 0x7f80e80cfcc0 2024-03-28T17:59:08.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80ef000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e80b3e70 con 0x7f80e8161c90 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80ef000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e8072c40 con 0x7f80e81617a0 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80edc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80e81617a0 0x7f80e8071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80edc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80e81617a0 0x7f80e8071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58346/0 (socket says 172.21.15.62:58346) 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.670+0000 7f80edc00640 1 -- 172.21.15.62:0/834601527 learned_addr learned my addr 172.21.15.62:0/834601527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.671+0000 7f80edc00640 1 -- 172.21.15.62:0/834601527 >> v1:172.21.15.81:6789/0 conn(0x7f80e8161c90 legacy=0x7f80e80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.671+0000 7f80edc00640 1 -- 172.21.15.62:0/834601527 >> v1:172.21.15.62:6789/0 conn(0x7f80e80cfcc0 legacy=0x7f80e80b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:08.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.671+0000 7f80edc00640 1 -- 172.21.15.62:0/834601527 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e80b4e70 con 0x7f80e81617a0 2024-03-28T17:59:08.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.672+0000 7f80edc00640 1 --2- 172.21.15.62:0/834601527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e81617a0 0x7f80e8071680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a98d3ab447f038b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80e7e00640 1 -- 172.21.15.62:0/834601527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc02f840 con 0x7f80e81617a0 2024-03-28T17:59:08.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80e7e00640 1 -- 172.21.15.62:0/834601527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc02f9e0 con 0x7f80e81617a0 2024-03-28T17:59:08.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80e7e00640 1 -- 172.21.15.62:0/834601527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc02fdb0 con 0x7f80e81617a0 2024-03-28T17:59:08.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 -- 172.21.15.62:0/834601527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e81617a0 msgr2=0x7f80e8071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 --2- 172.21.15.62:0/834601527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e81617a0 0x7f80e8071680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc02ba20 comp rx=0 tx=0).stop 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 -- 172.21.15.62:0/834601527 shutdown_connections 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 --2- 172.21.15.62:0/834601527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e81617a0 0x7f80e8071680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 -- 172.21.15.62:0/834601527 >> 172.21.15.62:0/834601527 conn(0x7f80e80b5790 msgr2=0x7f80e8072530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 -- 172.21.15.62:0/834601527 shutdown_connections 2024-03-28T17:59:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.673+0000 7f80ef000640 1 -- 172.21.15.62:0/834601527 wait complete. 2024-03-28T17:59:08.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.675+0000 7f80ef000640 1 Processor -- start 2024-03-28T17:59:08.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.675+0000 7f80ef000640 1 -- start start 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80e80cfcc0 0x7f80e815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 0x7f80e8148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8162910 con 0x7f80e8161c90 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e80731e0 con 0x7f80e81617a0 2024-03-28T17:59:08.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ef000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8162a90 con 0x7f80e80cfcc0 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.676+0000 7f80ee600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58354/0 (socket says 172.21.15.62:58354) 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 -- 172.21.15.62:0/2533321590 learned_addr learned my addr 172.21.15.62:0/2533321590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80edc00640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80e80cfcc0 0x7f80e815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ed200640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 0x7f80e8148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 -- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80e80cfcc0 msgr2=0x7f80e815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80e80cfcc0 0x7f80e815dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 -- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 msgr2=0x7f80e8148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 0x7f80e8148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 -- 172.21.15.62:0/2533321590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e81609b0 con 0x7f80e8161c90 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ed200640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 0x7f80e8148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:08.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.677+0000 7f80ee600640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c760 tx=0x7f80e000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.678+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0016020 con 0x7f80e8161c90 2024-03-28T17:59:08.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.678+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e0006d50 con 0x7f80e8161c90 2024-03-28T17:59:08.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.678+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0003bb0 con 0x7f80e8161c90 2024-03-28T17:59:08.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.678+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e81f8560 con 0x7f80e8161c90 2024-03-28T17:59:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.678+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81f8b20 con 0x7f80e8161c90 2024-03-28T17:59:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.679+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e804f400 con 0x7f80e8161c90 2024-03-28T17:59:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.680+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f80e00050d0 con 0x7f80e8161c90 2024-03-28T17:59:08.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.681+0000 7f80e5600640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 0x7f80c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:08.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.681+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80e008a600 con 0x7f80e8161c90 2024-03-28T17:59:08.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.681+0000 7f80edc00640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 0x7f80c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:08.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.682+0000 7f80edc00640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 0x7f80c806e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80dc036000 tx=0x7f80dc039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.685+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f80e0020020 con 0x7f80e8161c90 2024-03-28T17:59:08.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.884+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80e8159eb0 con 0x7f80e8161c90 2024-03-28T17:59:08.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.884+0000 7f80e5600640 1 -- 172.21.15.62:0/2533321590 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80e81609b0 con 0x7f80e8161c90 2024-03-28T17:59:08.893 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:08.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 msgr2=0x7f80c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 0x7f80c806e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80dc036000 tx=0x7f80dc039040 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 msgr2=0x7f80e81f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c760 tx=0x7f80e000cb60 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 shutdown_connections 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f80c806bbd0 0x7f80c806e090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80e8161c90 0x7f80e81f54a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e81617a0 0x7f80e8148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 --2- 172.21.15.62:0/2533321590 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80e80cfcc0 0x7f80e815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 >> 172.21.15.62:0/2533321590 conn(0x7f80e80b5790 msgr2=0x7f80e8152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 shutdown_connections 2024-03-28T17:59:08.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:08.887+0000 7f80ef000640 1 -- 172.21.15.62:0/2533321590 wait complete. 2024-03-28T17:59:08.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:09.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.148+0000 7fb947c00640 1 Processor -- start 2024-03-28T17:59:09.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.149+0000 7fb947c00640 1 -- start start 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.150+0000 7fb947c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb940072ad0 0x7fb940070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.150+0000 7fb947c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400b3470 con 0x7fb9400723f0 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.150+0000 7fb947c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9401626d0 con 0x7fb94015e030 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.150+0000 7fb947c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9401629b0 con 0x7fb940072ad0 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb940072ad0 0x7fb940070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb940072ad0 0x7fb940070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58360/0 (socket says 172.21.15.62:58360) 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 -- 172.21.15.62:0/477631572 learned_addr learned my addr 172.21.15.62:0/477631572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb945c00640 1 --1- 172.21.15.62:0/477631572 >> v1:172.21.15.62:6789/0 conn(0x7fb9400723f0 0x7fb9400710c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53754/0 (socket says 172.21.15.62:53754) 2024-03-28T17:59:09.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93f400640 1 -- 172.21.15.62:0/477631572 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526089302 0 0) 0x7fb9400b3470 con 0x7fb9400723f0 2024-03-28T17:59:09.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 -- 172.21.15.62:0/477631572 >> v1:172.21.15.81:6789/0 conn(0x7fb94015e030 legacy=0x7fb940161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:09.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 -- 172.21.15.62:0/477631572 >> v1:172.21.15.62:6789/0 conn(0x7fb9400723f0 legacy=0x7fb9400710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.151+0000 7fb93fe00640 1 -- 172.21.15.62:0/477631572 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94006cf30 con 0x7fb940072ad0 2024-03-28T17:59:09.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.152+0000 7fb93fe00640 1 --2- 172.21.15.62:0/477631572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940070b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb928009870 tx=0x7fb92802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab59305c2b8ff32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb93f400640 1 -- 172.21.15.62:0/477631572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928034070 con 0x7fb940072ad0 2024-03-28T17:59:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb93f400640 1 -- 172.21.15.62:0/477631572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928030860 con 0x7fb940072ad0 2024-03-28T17:59:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb93f400640 1 -- 172.21.15.62:0/477631572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928030c30 con 0x7fb940072ad0 2024-03-28T17:59:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 -- 172.21.15.62:0/477631572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 msgr2=0x7fb940070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 --2- 172.21.15.62:0/477631572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940070b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb928009870 tx=0x7fb92802b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 -- 172.21.15.62:0/477631572 shutdown_connections 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 --2- 172.21.15.62:0/477631572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940070b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 -- 172.21.15.62:0/477631572 >> 172.21.15.62:0/477631572 conn(0x7fb9400b1650 msgr2=0x7fb9400b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 -- 172.21.15.62:0/477631572 shutdown_connections 2024-03-28T17:59:09.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.153+0000 7fb947c00640 1 -- 172.21.15.62:0/477631572 wait complete. 2024-03-28T17:59:09.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.155+0000 7fb947c00640 1 Processor -- start 2024-03-28T17:59:09.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.155+0000 7fb947c00640 1 -- start start 2024-03-28T17:59:09.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9400723f0 0x7fb940152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb94015e030 0x7fb94014b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9401623e0 con 0x7fb940072ad0 2024-03-28T17:59:09.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400b37a0 con 0x7fb9400723f0 2024-03-28T17:59:09.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.156+0000 7fb947c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb940159640 con 0x7fb94015e030 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58376/0 (socket says 172.21.15.62:58376) 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 -- 172.21.15.62:0/61656464 learned_addr learned my addr 172.21.15.62:0/61656464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb945200640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9400723f0 0x7fb940152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 -- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb94015e030 msgr2=0x7fb94014b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb945c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb94015e030 0x7fb94014b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb94015e030 0x7fb94014b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 -- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9400723f0 msgr2=0x7fb940152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9400723f0 0x7fb940152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.157+0000 7fb93fe00640 1 -- 172.21.15.62:0/61656464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94014d910 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb93fe00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb928009870 tx=0x7fb928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928048070 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928030660 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928034040 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb940158a30 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.158+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb940158ff0 con 0x7fb940072ad0 2024-03-28T17:59:09.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.160+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94015db70 con 0x7fb940072ad0 2024-03-28T17:59:09.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.160+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb928037070 con 0x7fb940072ad0 2024-03-28T17:59:09.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.161+0000 7fb93cc00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 0x7fb91c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.161+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9280ac8f0 con 0x7fb940072ad0 2024-03-28T17:59:09.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.162+0000 7fb945200640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 0x7fb91c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.163+0000 7fb945200640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 0x7fb91c06e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb930004ea0 tx=0x7fb930009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.166+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb92807b3e0 con 0x7fb940072ad0 2024-03-28T17:59:09.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.367+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9400680c0 con 0x7fb940072ad0 2024-03-28T17:59:09.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.368+0000 7fb93cc00640 1 -- 172.21.15.62:0/61656464 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb92807ace0 con 0x7fb940072ad0 2024-03-28T17:59:09.368 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 msgr2=0x7fb91c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 0x7fb91c06e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb930004ea0 tx=0x7fb930009040 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 msgr2=0x7fb940159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb928009870 tx=0x7fb928004730 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 shutdown_connections 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb91c06bbd0 0x7fb91c06e090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb94015e030 0x7fb94014b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb940072ad0 0x7fb940159ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 --2- 172.21.15.62:0/61656464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9400723f0 0x7fb940152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.370+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 >> 172.21.15.62:0/61656464 conn(0x7fb9400b1650 msgr2=0x7fb9400aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:09.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.371+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 shutdown_connections 2024-03-28T17:59:09.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.371+0000 7fb947c00640 1 -- 172.21.15.62:0/61656464 wait complete. 2024-03-28T17:59:09.392 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T17:59:09.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.614+0000 7f9f31e00640 1 Processor -- start 2024-03-28T17:59:09.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.614+0000 7f9f31e00640 1 -- start start 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.615+0000 7f9f31e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 0x7f9f2c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.615+0000 7f9f31e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.615+0000 7f9f31e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c1603c0 con 0x7f9f2c15fa00 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.615+0000 7f9f31e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c1606a0 con 0x7f9f2c0cf590 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.615+0000 7f9f31e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c0b3cb0 con 0x7f9f2c152f90 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2a000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9f2c15fa00 0x7f9f2c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53758/0 (socket says 172.21.15.62:53758) 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58388/0 (socket says 172.21.15.62:58388) 2024-03-28T17:59:09.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2a000640 1 -- 172.21.15.62:0/3399598690 learned_addr learned my addr 172.21.15.62:0/3399598690 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2aa00640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 0x7f9f2c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 -- 172.21.15.62:0/3399598690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 msgr2=0x7f9f2c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 0x7f9f2c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 -- 172.21.15.62:0/3399598690 >> v1:172.21.15.62:6789/0 conn(0x7f9f2c15fa00 legacy=0x7f9f2c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2b400640 1 -- 172.21.15.62:0/3399598690 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0b3f90 con 0x7f9f2c0cf590 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.616+0000 7f9f2aa00640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 0x7f9f2c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.617+0000 7f9f2b400640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009d90 tx=0x7f9f2402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727e9797e02f29d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f29600640 1 -- 172.21.15.62:0/3399598690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24039070 con 0x7f9f2c0cf590 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f29600640 1 -- 172.21.15.62:0/3399598690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2402cc60 con 0x7f9f2c0cf590 2024-03-28T17:59:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f29600640 1 -- 172.21.15.62:0/3399598690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24034660 con 0x7f9f2c0cf590 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 -- 172.21.15.62:0/3399598690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c0cf590 msgr2=0x7f9f2c0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009d90 tx=0x7f9f2402beb0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 -- 172.21.15.62:0/3399598690 shutdown_connections 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c0b44b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 --2- 172.21.15.62:0/3399598690 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c152f90 0x7f9f2c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 -- 172.21.15.62:0/3399598690 >> 172.21.15.62:0/3399598690 conn(0x7f9f2c066fb0 msgr2=0x7f9f2c0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:09.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 -- 172.21.15.62:0/3399598690 shutdown_connections 2024-03-28T17:59:09.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.618+0000 7f9f31e00640 1 -- 172.21.15.62:0/3399598690 wait complete. 2024-03-28T17:59:09.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.620+0000 7f9f31e00640 1 Processor -- start 2024-03-28T17:59:09.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.620+0000 7f9f31e00640 1 -- start start 2024-03-28T17:59:09.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c15fa00 0x7f9f2c13fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c160000 con 0x7f9f2c152f90 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c15fe80 con 0x7f9f2c0cf590 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f31e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c160180 con 0x7f9f2c15fa00 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58396/0 (socket says 172.21.15.62:58396) 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.621+0000 7f9f2a000640 1 -- 172.21.15.62:0/1118602473 learned_addr learned my addr 172.21.15.62:0/1118602473 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2b400640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c15fa00 0x7f9f2c13fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2aa00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 -- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c15fa00 msgr2=0x7f9f2c13fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c15fa00 0x7f9f2c13fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 -- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2c0cf590 msgr2=0x7f9f2c072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 -- 172.21.15.62:0/1118602473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c09c070 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f2a000640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c3f0 tx=0x7f9f1800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18017070 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18005ce0 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180072f0 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.622+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c09c350 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.623+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c09c910 con 0x7f9f2c152f90 2024-03-28T17:59:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.624+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c0680c0 con 0x7f9f2c152f90 2024-03-28T17:59:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.625+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9f1801a070 con 0x7f9f2c152f90 2024-03-28T17:59:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.625+0000 7f9f23400640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 0x7f9f0c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.626+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f18089e10 con 0x7f9f2c152f90 2024-03-28T17:59:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.626+0000 7f9f2aa00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 0x7f9f0c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:09.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.627+0000 7f9f2aa00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 0x7f9f0c06e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d8b0 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:09.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.630+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9f1808d030 con 0x7f9f2c152f90 2024-03-28T17:59:09.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.871+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f2c0b3500 con 0x7f9f2c152f90 2024-03-28T17:59:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.872+0000 7f9f23400640 1 -- 172.21.15.62:0/1118602473 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f9f180588f0 con 0x7f9f2c152f90 2024-03-28T17:59:09.873 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:09.873 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T17:59:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 msgr2=0x7f9f0c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 0x7f9f0c06e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d8b0 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 msgr2=0x7f9f2c142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c3f0 tx=0x7f9f1800c8b0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 shutdown_connections 2024-03-28T17:59:09.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f0c06bbd0 0x7f9f0c06e090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9f2c15fa00 0x7f9f2c13fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c152f90 0x7f9f2c142bb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 --2- 172.21.15.62:0/1118602473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2c0cf590 0x7f9f2c072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 >> 172.21.15.62:0/1118602473 conn(0x7f9f2c066fb0 msgr2=0x7f9f2c0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 shutdown_connections 2024-03-28T17:59:09.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:09.875+0000 7f9f31e00640 1 -- 172.21.15.62:0/1118602473 wait complete. 2024-03-28T17:59:09.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.113+0000 7fad90a00640 1 Processor -- start 2024-03-28T17:59:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.114+0000 7fad90a00640 1 -- start start 2024-03-28T17:59:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad90a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c15dd60 0x7fad8c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad90a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad8c0b99f0 0x7fad8c0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad90a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0b6e60 con 0x7fad8c0b5c30 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad90a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0b60a0 con 0x7fad8c0b99f0 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad90a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b6c30 con 0x7fad8c15dd60 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad8be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fad8c0b5c30 0x7fad8c161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53770/0 (socket says 172.21.15.62:53770) 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.115+0000 7fad8be00640 1 -- 172.21.15.62:0/73915325 learned_addr learned my addr 172.21.15.62:0/73915325 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad83e00640 1 --2- 172.21.15.62:0/73915325 >> v2:172.21.15.62:3300/0 conn(0x7fad8c0b99f0 0x7fad8c0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad8b400640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c15dd60 0x7fad8c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad8aa00640 1 -- 172.21.15.62:0/73915325 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160723532 0 0) 0x7fad8c0b6e60 con 0x7fad8c0b5c30 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad8aa00640 1 -- 172.21.15.62:0/73915325 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad70003520 con 0x7fad8c0b5c30 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad83e00640 1 -- 172.21.15.62:0/73915325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c15dd60 msgr2=0x7fad8c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad83e00640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c15dd60 0x7fad8c0b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad83e00640 1 -- 172.21.15.62:0/73915325 >> v1:172.21.15.62:6789/0 conn(0x7fad8c0b5c30 legacy=0x7fad8c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.116+0000 7fad83e00640 1 -- 172.21.15.62:0/73915325 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0b7100 con 0x7fad8c0b99f0 2024-03-28T17:59:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.117+0000 7fad83e00640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b99f0 0x7fad8c0b56f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbefe79556f2261 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad8aa00640 1 -- 172.21.15.62:0/73915325 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74034070 con 0x7fad8c0b99f0 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad8aa00640 1 -- 172.21.15.62:0/73915325 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74030800 con 0x7fad8c0b99f0 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad8aa00640 1 -- 172.21.15.62:0/73915325 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74030bd0 con 0x7fad8c0b99f0 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 -- 172.21.15.62:0/73915325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b99f0 msgr2=0x7fad8c0b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b99f0 0x7fad8c0b56f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b940 comp rx=0 tx=0).stop 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 -- 172.21.15.62:0/73915325 shutdown_connections 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b99f0 0x7fad8c0b56f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 --2- 172.21.15.62:0/73915325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c15dd60 0x7fad8c0b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 -- 172.21.15.62:0/73915325 >> 172.21.15.62:0/73915325 conn(0x7fad8c071ae0 msgr2=0x7fad8c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 -- 172.21.15.62:0/73915325 shutdown_connections 2024-03-28T17:59:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.118+0000 7fad90a00640 1 -- 172.21.15.62:0/73915325 wait complete. 2024-03-28T17:59:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.120+0000 7fad90a00640 1 Processor -- start 2024-03-28T17:59:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.120+0000 7fad90a00640 1 -- start start 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c0b99f0 0x7fad8c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad8c15dd60 0x7fad8c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b79a0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b76c0 con 0x7fad8c15dd60 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad90a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c15e140 con 0x7fad8c0b99f0 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad8b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad8b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58412/0 (socket says 172.21.15.62:58412) 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.121+0000 7fad8b400640 1 -- 172.21.15.62:0/394597947 learned_addr learned my addr 172.21.15.62:0/394597947 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad83e00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c0b99f0 0x7fad8c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8be00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad8c15dd60 0x7fad8c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 -- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c0b99f0 msgr2=0x7fad8c14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c0b99f0 0x7fad8c14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 -- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad8c15dd60 msgr2=0x7fad8c0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad8c15dd60 0x7fad8c0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 -- 172.21.15.62:0/394597947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c1f4ec0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad8b400640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d5f0 tx=0x7fad7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0125f0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c019070 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0129d0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.122+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c1f4300 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.123+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1f48c0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.124+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c072b90 con 0x7fad8c0b5c30 2024-03-28T17:59:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.125+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fad7c012b70 con 0x7fad8c0b5c30 2024-03-28T17:59:10.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.126+0000 7fad83400640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 0x7fad6806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.126+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c08aed0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.126+0000 7fad83e00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 0x7fad6806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.127+0000 7fad83e00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 0x7fad6806e0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.130+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fad7c059920 con 0x7fad8c0b5c30 2024-03-28T17:59:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.334+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad8c04f4e0 con 0x7fad8c0b5c30 2024-03-28T17:59:10.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.337+0000 7fad83400640 1 -- 172.21.15.62:0/394597947 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fad7c059220 con 0x7fad8c0b5c30 2024-03-28T17:59:10.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:10.338 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:10.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.339+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 msgr2=0x7fad6806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.339+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 0x7fad6806e0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403b040 comp rx=0 tx=0).stop 2024-03-28T17:59:10.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 msgr2=0x7fad8c1f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d5f0 tx=0x7fad7c00dab0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 shutdown_connections 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fad6806bc00 0x7fad6806e0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad8c15dd60 0x7fad8c0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fad8c0b99f0 0x7fad8c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 --2- 172.21.15.62:0/394597947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad8c0b5c30 0x7fad8c1f3ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 >> 172.21.15.62:0/394597947 conn(0x7fad8c071ae0 msgr2=0x7fad8c15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 shutdown_connections 2024-03-28T17:59:10.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.340+0000 7fad90a00640 1 -- 172.21.15.62:0/394597947 wait complete. 2024-03-28T17:59:10.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T17:59:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.578+0000 7f3eb9e00640 1 Processor -- start 2024-03-28T17:59:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.579+0000 7f3eb9e00640 1 -- start start 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 0x7f3eb40ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3eb4071d10 0x7f3eb40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40d3330 con 0x7f3eb4071d10 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40d24a0 con 0x7f3eb40b3310 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.580+0000 7f3eb9e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d2720 con 0x7f3eb4071930 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3eb4071d10 0x7f3eb40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3eb4071d10 0x7f3eb40cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58424/0 (socket says 172.21.15.62:58424) 2024-03-28T17:59:10.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 -- 172.21.15.62:0/3305440512 learned_addr learned my addr 172.21.15.62:0/3305440512 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eabe00640 1 --2- 172.21.15.62:0/3305440512 >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 0x7f3eb40ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb2a00640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 -- 172.21.15.62:0/3305440512 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 msgr2=0x7f3eb40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 -- 172.21.15.62:0/3305440512 >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 msgr2=0x7f3eb40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 --2- 172.21.15.62:0/3305440512 >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 0x7f3eb40ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eb3400640 1 -- 172.21.15.62:0/3305440512 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40d2a00 con 0x7f3eb4071d10 2024-03-28T17:59:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.581+0000 7f3eabe00640 1 --2- 172.21.15.62:0/3305440512 >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 0x7f3eb40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.582+0000 7f3eb3400640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb40cf260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3eb408c5d0 tx=0x7f3eac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9352aa446308a7f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb2000640 1 -- 172.21.15.62:0/3305440512 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac02cb00 con 0x7f3eb4071d10 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb2000640 1 -- 172.21.15.62:0/3305440512 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac02cca0 con 0x7f3eb4071d10 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb2000640 1 -- 172.21.15.62:0/3305440512 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac034610 con 0x7f3eb4071d10 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3305440512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 msgr2=0x7f3eb40cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb40cf260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3eb408c5d0 tx=0x7f3eac02bf60 comp rx=0 tx=0).stop 2024-03-28T17:59:10.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.583+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3305440512 shutdown_connections 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb40cf260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3305440512 >> v2:172.21.15.81:3300/0 conn(0x7f3eb40b3310 0x7f3eb40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3305440512 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3305440512 >> 172.21.15.62:0/3305440512 conn(0x7f3eb40d3ef0 msgr2=0x7f3eb40d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3305440512 shutdown_connections 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.584+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3305440512 wait complete. 2024-03-28T17:59:10.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.586+0000 7f3eb9e00640 1 Processor -- start 2024-03-28T17:59:10.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.586+0000 7f3eb9e00640 1 -- start start 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3eb40b3310 0x7f3eb415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d4dd0 con 0x7f3eb4071d10 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d2fc0 con 0x7f3eb40b3310 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eb9e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d37c0 con 0x7f3eb4071930 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58430/0 (socket says 172.21.15.62:58430) 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.587+0000 7f3eabe00640 1 -- 172.21.15.62:0/3891031557 learned_addr learned my addr 172.21.15.62:0/3891031557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eb3400640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3eb40b3310 0x7f3eb415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eb2a00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 -- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 msgr2=0x7f3eb40b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 -- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3eb40b3310 msgr2=0x7f3eb415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3eb40b3310 0x7f3eb415b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 -- 172.21.15.62:0/3891031557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40b64f0 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eabe00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e7e0 tx=0x7f3e9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00f040 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c004030 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40b67b0 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004400 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.588+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb40c8d70 con 0x7f3eb4071d10 2024-03-28T17:59:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.589+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb40680c0 con 0x7f3eb4071d10 2024-03-28T17:59:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.591+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3e9c00c6d0 con 0x7f3eb4071d10 2024-03-28T17:59:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.591+0000 7f3eab400640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 0x7f3e9006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.592+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e9c08a900 con 0x7f3eb4071d10 2024-03-28T17:59:10.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.592+0000 7f3eb2a00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 0x7f3e9006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:10.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.593+0000 7f3eb2a00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 0x7f3e9006e090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006fd0 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.596+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3e9c017020 con 0x7f3eb4071d10 2024-03-28T17:59:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.837+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb40b9710 con 0x7f3eb4071d10 2024-03-28T17:59:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.838+0000 7f3eab400640 1 -- 172.21.15.62:0/3891031557 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f3e9c059350 con 0x7f3eb4071d10 2024-03-28T17:59:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:10.839 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T17:59:10.839 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 msgr2=0x7f3e9006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 0x7f3e9006e090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006fd0 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 msgr2=0x7f3eb41e7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e7e0 tx=0x7f3e9c00eca0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 shutdown_connections 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3e9006bbd0 0x7f3e9006e090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3eb40b3310 0x7f3eb415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eb4071d10 0x7f3eb41e7350 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.841+0000 7f3eb9e00640 1 --2- 172.21.15.62:0/3891031557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3eb4071930 0x7f3eb40b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.842+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 >> 172.21.15.62:0/3891031557 conn(0x7f3eb40d3ef0 msgr2=0x7f3eb415e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.842+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 shutdown_connections 2024-03-28T17:59:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:10.842+0000 7f3eb9e00640 1 -- 172.21.15.62:0/3891031557 wait complete. 2024-03-28T17:59:11.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T17:58:53.451830+0000', 'modified': '2024-03-28T17:58:56.522056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6837/3525951990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3525951990}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3525951990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4297, 'qdb_cluster': [4297]} 2024-03-28T17:59:11.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T17:59:11.263 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-28T17:59:11.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.489+0000 7f5b92000640 1 Processor -- start 2024-03-28T17:59:11.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.490+0000 7f5b92000640 1 -- start start 2024-03-28T17:59:11.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.491+0000 7f5b92000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5b8c154fa0 0x7f5b8c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.491+0000 7f5b92000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c0ae3e0 con 0x7f5b8c071c30 2024-03-28T17:59:11.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.491+0000 7f5b92000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c0b5250 con 0x7f5b8c0b4240 2024-03-28T17:59:11.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.491+0000 7f5b92000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8c0b49b0 con 0x7f5b8c154fa0 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.492+0000 7f5b8b400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5b8c154fa0 0x7f5b8c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.492+0000 7f5b8be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5b8c071c30 0x7f5b8c0aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53774/0 (socket says 172.21.15.62:53774) 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.492+0000 7f5b8be00640 1 -- 172.21.15.62:0/3577898575 learned_addr learned my addr 172.21.15.62:0/3577898575 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.492+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829385592 0 0) 0x7f5b84001490 con 0x7f5b8c071c30 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.492+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b70003520 con 0x7f5b8c071c30 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.493+0000 7f5b8b400640 1 -- 172.21.15.62:0/3577898575 >> v1:172.21.15.81:6789/0 conn(0x7f5b8c0b4240 legacy=0x7f5b8c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.493+0000 7f5b8b400640 1 -- 172.21.15.62:0/3577898575 >> v1:172.21.15.62:6789/0 conn(0x7f5b8c071c30 legacy=0x7f5b8c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.493+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111555698 0 0) 0x7f5b70003520 con 0x7f5b8c071c30 2024-03-28T17:59:11.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.493+0000 7f5b8b400640 1 -- 172.21.15.62:0/3577898575 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0b4c90 con 0x7f5b8c154fa0 2024-03-28T17:59:11.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.494+0000 7f5b8b400640 1 --2- 172.21.15.62:0/3577898575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c154fa0 0x7f5b8c15e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d2bbe7bd14cca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.494+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037070 con 0x7f5b8c154fa0 2024-03-28T17:59:11.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c030510 con 0x7f5b8c154fa0 2024-03-28T17:59:11.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b8a000640 1 -- 172.21.15.62:0/3577898575 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84001bc0 con 0x7f5b8c154fa0 2024-03-28T17:59:11.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b92000640 1 -- 172.21.15.62:0/3577898575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c154fa0 msgr2=0x7f5b8c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b92000640 1 --2- 172.21.15.62:0/3577898575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c154fa0 0x7f5b8c15e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02b790 comp rx=0 tx=0).stop 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b92000640 1 -- 172.21.15.62:0/3577898575 shutdown_connections 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b92000640 1 --2- 172.21.15.62:0/3577898575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c154fa0 0x7f5b8c15e650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.495+0000 7f5b92000640 1 -- 172.21.15.62:0/3577898575 >> 172.21.15.62:0/3577898575 conn(0x7f5b8c0671f0 msgr2=0x7f5b8c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.496+0000 7f5b92000640 1 -- 172.21.15.62:0/3577898575 shutdown_connections 2024-03-28T17:59:11.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.496+0000 7f5b92000640 1 -- 172.21.15.62:0/3577898575 wait complete. 2024-03-28T17:59:11.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.497+0000 7f5b92000640 1 Processor -- start 2024-03-28T17:59:11.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.497+0000 7f5b92000640 1 -- start start 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c071c30 0x7f5b8c162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b8c154fa0 0x7f5b8c15f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0b46d0 con 0x7f5b8c0b4240 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0ae660 con 0x7f5b8c071c30 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.498+0000 7f5b92000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c155380 con 0x7f5b8c154fa0 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58442/0 (socket says 172.21.15.62:58442) 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 -- 172.21.15.62:0/4220988097 learned_addr learned my addr 172.21.15.62:0/4220988097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8be00640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b8c154fa0 0x7f5b8c15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c071c30 0x7f5b8c162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 -- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b8c154fa0 msgr2=0x7f5b8c15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b8c154fa0 0x7f5b8c15f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 -- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c071c30 msgr2=0x7f5b8c162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c071c30 0x7f5b8c162230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 -- 172.21.15.62:0/4220988097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c15fe10 con 0x7f5b8c0b4240 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.499+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c400 tx=0x7f5b7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.500+0000 7f5b83400640 1 -- 172.21.15.62:0/4220988097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740052d0 con 0x7f5b8c0b4240 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.500+0000 7f5b83400640 1 -- 172.21.15.62:0/4220988097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74005490 con 0x7f5b8c0b4240 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.500+0000 7f5b83400640 1 -- 172.21.15.62:0/4220988097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74007260 con 0x7f5b8c0b4240 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.500+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c1600f0 con 0x7f5b8c0b4240 2024-03-28T17:59:11.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.500+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8c1606b0 con 0x7f5b8c0b4240 2024-03-28T17:59:11.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.502+0000 7f5b83400640 1 -- 172.21.15.62:0/4220988097 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5b74007400 con 0x7f5b8c0b4240 2024-03-28T17:59:11.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.503+0000 7f5b83400640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 0x7f5b6806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.503+0000 7f5b83400640 1 -- 172.21.15.62:0/4220988097 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b74089ff0 con 0x7f5b8c0b4240 2024-03-28T17:59:11.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.503+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 0x7f5b6806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.504+0000 7f5b94df1740 1 Processor -- start 2024-03-28T17:59:11.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.504+0000 7f5b94df1740 1 -- start start 2024-03-28T17:59:11.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 0x7f5b6806e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c034000 tx=0x7f5b7c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc8cbd0 0x56010fc8cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc5cbf0 0x56010fc89e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fbc6c90 con 0x56010fc60750 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fb6f980 con 0x56010fc8cbd0 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010fbc2d30 con 0x56010fc5cbf0 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58456/0 (socket says 172.21.15.62:58456) 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 -- 172.21.15.62:0/2492064733 learned_addr learned my addr 172.21.15.62:0/2492064733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8b400640 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc8cbd0 0x56010fc8cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc5cbf0 0x56010fc89e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 -- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc5cbf0 msgr2=0x56010fc89e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc5cbf0 0x56010fc89e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 -- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc8cbd0 msgr2=0x56010fc8cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc8cbd0 0x56010fc8cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.505+0000 7f5b8be00640 1 -- 172.21.15.62:0/2492064733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010fb87510 con 0x56010fc60750 2024-03-28T17:59:11.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.506+0000 7f5b8be00640 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b840016f0 tx=0x7f5b8400e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fadb5c68c3dcfdc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.506+0000 7f5b7b400640 1 -- 172.21.15.62:0/2492064733 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84019070 con 0x56010fc60750 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.506+0000 7f5b7b400640 1 -- 172.21.15.62:0/2492064733 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84014480 con 0x56010fc60750 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.506+0000 7f5b7b400640 1 -- 172.21.15.62:0/2492064733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84014870 con 0x56010fc60750 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 -- 172.21.15.62:0/2492064733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 msgr2=0x56010fc8a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b840016f0 tx=0x7f5b8400e5f0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 -- 172.21.15.62:0/2492064733 shutdown_connections 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fc8a380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc5cbf0 0x56010fc89e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 --2- 172.21.15.62:0/2492064733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc8cbd0 0x56010fc8cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 -- 172.21.15.62:0/2492064733 >> 172.21.15.62:0/2492064733 conn(0x56010fc5ee60 msgr2=0x56010fca79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 -- 172.21.15.62:0/2492064733 shutdown_connections 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.507+0000 7f5b94df1740 1 -- 172.21.15.62:0/2492064733 wait complete. 2024-03-28T17:59:11.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.508+0000 7f5b94df1740 1 -- start start 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.508+0000 7f5b94df1740 1 Processor -- start 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc8cbd0 0x56010fc820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fc860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fc61900 con 0x56010fc60750 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fb82dd0 con 0x56010fc826f0 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010fc60cf0 con 0x56010fc8cbd0 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b94df1740 10 client.0 _ll_register_callbacks cb 0x56010f8daac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b8b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b8b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58464/0 (socket says 172.21.15.62:58464) 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b8b400640 1 -- 172.21.15.62:0/4146511634 learned_addr learned my addr 172.21.15.62:0/4146511634 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.509+0000 7f5b8be00640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fc860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc8cbd0 0x56010fc820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 -- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc8cbd0 msgr2=0x56010fc820d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc8cbd0 0x56010fc820d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 -- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 msgr2=0x56010fc860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fc860e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 -- 172.21.15.62:0/4146511634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010f8db2d0 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b8b400640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02bda0 tx=0x7f5b7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2858749eed7c44b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037040 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b7c04a680 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.510+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b84014480 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b7c04a880 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 10 client.4653 fetch_fsmap learned FSMap version 8 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56010f9a56b0 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84014480 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f5b7c04aa20 con 0x56010fc60750 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 10 client.4653 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 10 client.4653 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-28T17:59:11.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 20 client.4653 populate_metadata read hostname 'smithi062' 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fcb7c00 0x56010fcb8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 4 client.4653 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.511+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 --> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010fb68520 con 0x56010fcb7c00 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.512+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fcb7c00 0x56010fcb8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.512+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fcb7c00 0x56010fcb8000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8d7fdf9f575723 server_cookie=2df3f5491f09f489 in_seq=0 out_seq=0 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.512+0000 7f5b6fe00640 10 client.4653 ms_handle_connect on v2:172.21.15.81:6834/3525951990 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b6fe00640 1 -- 172.21.15.62:0/4146511634 <== mds.0 v2:172.21.15.81:6834/3525951990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56010fb68520 con 0x56010fcb7c00 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b6fe00640 10 client.4653 handle_command_reply: tid=0 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 client.4653 shutdown 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 msgr2=0x56010fcbbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02bda0 tx=0x7f5b7c004730 comp rx=0 tx=0).stop 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 shutdown_connections 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fcb7c00 0x56010fcb8000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fc860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc8cbd0 0x56010fc820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 --2- 172.21.15.62:0/4146511634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc60750 0x56010fcbbbe0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b6fe00640 0 client.4653 ms_handle_reset on v2:172.21.15.81:6834/3525951990 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 >> 172.21.15.62:0/4146511634 conn(0x56010fc5cbf0 msgr2=0x56010fcaaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 shutdown_connections 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.514+0000 7f5b94df1740 1 -- 172.21.15.62:0/4146511634 wait complete. 2024-03-28T17:59:11.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.515+0000 7f5b94df1740 20 client.4653 trim_cache size 0 max 16384 2024-03-28T17:59:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.534+0000 7f5b94df1740 1 Processor -- start 2024-03-28T17:59:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.534+0000 7f5b94df1740 1 -- start start 2024-03-28T17:59:11.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc60750 0x56010fc8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc826f0 0x56010fcaaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c1ff410 con 0x56010fc8cbd0 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c1f43b0 con 0x56010fc60750 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c1f49b0 con 0x56010fc826f0 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58470/0 (socket says 172.21.15.62:58470) 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b8be00640 1 -- 172.21.15.62:0/320364013 learned_addr learned my addr 172.21.15.62:0/320364013 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b8b400640 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc60750 0x56010fc8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.535+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc826f0 0x56010fcaaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 -- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc826f0 msgr2=0x56010fcaaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc826f0 0x56010fcaaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 -- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc60750 msgr2=0x56010fc8c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc60750 0x56010fc8c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 -- 172.21.15.62:0/320364013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010fc5ec00 con 0x56010fc8cbd0 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b8be00640 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009010 tx=0x7f5b8400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83a0402f0d6829f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b7aa00640 1 -- 172.21.15.62:0/320364013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84019040 con 0x56010fc8cbd0 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b7aa00640 1 -- 172.21.15.62:0/320364013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84024070 con 0x56010fc8cbd0 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.536+0000 7f5b7aa00640 1 -- 172.21.15.62:0/320364013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840148e0 con 0x56010fc8cbd0 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 -- 172.21.15.62:0/320364013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 msgr2=0x56010fcab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009010 tx=0x7f5b8400a000 comp rx=0 tx=0).stop 2024-03-28T17:59:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 -- 172.21.15.62:0/320364013 shutdown_connections 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fcab430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc826f0 0x56010fcaaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 --2- 172.21.15.62:0/320364013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc60750 0x56010fc8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 -- 172.21.15.62:0/320364013 >> 172.21.15.62:0/320364013 conn(0x56010fc5cbf0 msgr2=0x56010fc782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 -- 172.21.15.62:0/320364013 shutdown_connections 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.537+0000 7f5b94df1740 1 -- 172.21.15.62:0/320364013 wait complete. 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.538+0000 7f5b94df1740 1 -- start start 2024-03-28T17:59:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.538+0000 7f5b94df1740 1 Processor -- start 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc60750 0x56010fcab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fd19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fc86760 con 0x56010fc8cbd0 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010fc86c20 con 0x56010fc826f0 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010fc8b350 con 0x56010fc60750 2024-03-28T17:59:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b94df1740 10 client.0 _ll_register_callbacks cb 0x7f5b7c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58472/0 (socket says 172.21.15.62:58472) 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 -- 172.21.15.62:0/1565358561 learned_addr learned my addr 172.21.15.62:0/1565358561 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fd19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8b400640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc60750 0x56010fcab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 -- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc60750 msgr2=0x56010fcab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc60750 0x56010fcab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 -- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 msgr2=0x56010fd19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.539+0000 7f5b8be00640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fd19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b8be00640 1 -- 172.21.15.62:0/1565358561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010fc8fb60 con 0x56010fc8cbd0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b8aa00640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fd19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b8be00640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b84001ba0 tx=0x7f5b840020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514f5f829ed2bd39 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-28T17:59:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84014890 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b84014a30 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b84024070 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.540+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b84035050 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 10 client.4659 fetch_fsmap learned FSMap version 8 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56010fc8fe30 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84024070 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f5b84038020 con 0x56010fc8cbd0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 10 client.4659 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 10 client.4659 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 20 client.4659 populate_metadata read hostname 'smithi062' 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fd15b50 0x56010fd15f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 4 client.4659 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.541+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 --> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56010fbd41d0 con 0x56010fd15b50 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.542+0000 7f5b8b400640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fd15b50 0x56010fd15f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.542+0000 7f5b8b400640 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fd15b50 0x56010fd15f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dc44626aadb5a server_cookie=208170be221c7590 in_seq=0 out_seq=0 2024-03-28T17:59:11.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.542+0000 7f5b6fe00640 10 client.4659 ms_handle_connect on v2:172.21.15.81:6834/3525951990 2024-03-28T17:59:11.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b6fe00640 1 -- 172.21.15.62:0/1565358561 <== mds.0 v2:172.21.15.81:6834/3525951990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56010fbd41d0 con 0x56010fd15b50 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b6fe00640 10 client.4659 handle_command_reply: tid=0 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 client.4659 shutdown 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 msgr2=0x56010fd1a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b84001ba0 tx=0x7f5b840020a0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 shutdown_connections 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:6834/3525951990,v1:172.21.15.81:6837/3525951990] conn(0x56010fd15b50 0x56010fd15f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56010fc8cbd0 0x56010fd1a1d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56010fc826f0 0x56010fd19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 --2- 172.21.15.62:0/1565358561 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56010fc60750 0x56010fcab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b6fe00640 0 client.4659 ms_handle_reset on v2:172.21.15.81:6834/3525951990 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 >> 172.21.15.62:0/1565358561 conn(0x56010fc5cbf0 msgr2=0x56010fc7ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 shutdown_connections 2024-03-28T17:59:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.543+0000 7f5b94df1740 1 -- 172.21.15.62:0/1565358561 wait complete. 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b94df1740 20 client.4659 trim_cache size 0 max 16384 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "aedb2108-ca9d-4a87-ad07-9f4b969c6f45", 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4297, 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 17.539604685, 2024-03-28T17:59:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 8, 2024-03-28T17:59:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 20, 2024-03-28T17:59:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 20, 2024-03-28T17:59:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 23.021463089000001 2024-03-28T17:59:11.548 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T17:59:11.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 msgr2=0x7f5b6806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 0x7f5b6806e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c034000 tx=0x7f5b7c02f000 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 msgr2=0x7f5b8c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c400 tx=0x7f5b7400c8c0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 shutdown_connections 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6806bbd0 0x7f5b6806e090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b8c154fa0 0x7f5b8c15f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0b4240 0x7f5b8c158d30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 --2- 172.21.15.62:0/4220988097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b8c071c30 0x7f5b8c162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 >> 172.21.15.62:0/4220988097 conn(0x7f5b8c0671f0 msgr2=0x7f5b8c071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 shutdown_connections 2024-03-28T17:59:11.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:11.544+0000 7f5b92000640 1 -- 172.21.15.62:0/4220988097 wait complete. 2024-03-28T17:59:11.562 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-28T17:59:11.588 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-28T17:59:11.588 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-28T17:59:11.588 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-28T17:59:11.588 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T17:59:11.588 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:11.588 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:11.589 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:11.589 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T17:59:11.589 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:11.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:11.589 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-03-28T17:59:11.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:11.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-03-28T17:59:11.750 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-03-28T17:59:11.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:59:11.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:11.755 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-28T17:59:11.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:11.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-28T17:59:11.894 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-03-28T17:59:11.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:59:11.898 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-28T17:59:11.898 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T17:59:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.128+0000 7f4740200640 1 Processor -- start 2024-03-28T17:59:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.128+0000 7f4740200640 1 -- start start 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4738144740 0x7f47380b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f473815ed90 0x7f47380ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47380b3700 con 0x7f473815ed90 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47380b39e0 con 0x7f4738144740 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f4740200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380b3cc0 con 0x7f47380b31f0 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f473e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f473815ed90 0x7f47380ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f473e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f473815ed90 0x7f47380ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58482/0 (socket says 172.21.15.62:58482) 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.130+0000 7f473e200640 1 -- 172.21.15.62:0/1601025477 learned_addr learned my addr 172.21.15.62:0/1601025477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473ce00640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473d800640 1 --2- 172.21.15.62:0/1601025477 >> v2:172.21.15.81:3300/0 conn(0x7f4738144740 0x7f47380b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473e200640 1 -- 172.21.15.62:0/1601025477 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 msgr2=0x7f473815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473e200640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473e200640 1 -- 172.21.15.62:0/1601025477 >> v2:172.21.15.81:3300/0 conn(0x7f4738144740 msgr2=0x7f47380b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473e200640 1 --2- 172.21.15.62:0/1601025477 >> v2:172.21.15.81:3300/0 conn(0x7f4738144740 0x7f47380b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.131+0000 7f473e200640 1 -- 172.21.15.62:0/1601025477 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380b3fa0 con 0x7f473815ed90 2024-03-28T17:59:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.132+0000 7f473e200640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f47380ae680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f473806ebe0 tx=0x7f473002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96873961b5762247 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.132+0000 7f4737e00640 1 -- 172.21.15.62:0/1601025477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730035020 con 0x7f473815ed90 2024-03-28T17:59:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.132+0000 7f4737e00640 1 -- 172.21.15.62:0/1601025477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4730031870 con 0x7f473815ed90 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.132+0000 7f4737e00640 1 -- 172.21.15.62:0/1601025477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730031c40 con 0x7f473815ed90 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 -- 172.21.15.62:0/1601025477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 msgr2=0x7f47380ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f47380ae680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f473806ebe0 tx=0x7f473002b910 comp rx=0 tx=0).stop 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 -- 172.21.15.62:0/1601025477 shutdown_connections 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f47380ae680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 --2- 172.21.15.62:0/1601025477 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 --2- 172.21.15.62:0/1601025477 >> v2:172.21.15.81:3300/0 conn(0x7f4738144740 0x7f47380b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 -- 172.21.15.62:0/1601025477 >> 172.21.15.62:0/1601025477 conn(0x7f4738066fb0 msgr2=0x7f47380673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 -- 172.21.15.62:0/1601025477 shutdown_connections 2024-03-28T17:59:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.133+0000 7f4740200640 1 -- 172.21.15.62:0/1601025477 wait complete. 2024-03-28T17:59:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.135+0000 7f4740200640 1 Processor -- start 2024-03-28T17:59:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.135+0000 7f4740200640 1 -- start start 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4738144740 0x7f47381f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380aed40 con 0x7f473815ed90 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380aebc0 con 0x7f4738144740 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f4740200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380aeec0 con 0x7f47380b31f0 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f473e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f473e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58498/0 (socket says 172.21.15.62:58498) 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.136+0000 7f473e200640 1 -- 172.21.15.62:0/3988757236 learned_addr learned my addr 172.21.15.62:0/3988757236 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473d800640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473ce00640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4738144740 0x7f47381f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 -- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 msgr2=0x7f473815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 -- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4738144740 msgr2=0x7f47381f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4738144740 0x7f47381f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 -- 172.21.15.62:0/3988757236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380afec0 con 0x7f473815ed90 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f473e200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f47300090b0 tx=0x7f47300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.137+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730040070 con 0x7f473815ed90 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.138+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4730030c50 con 0x7f473815ed90 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.138+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380b01a0 con 0x7f473815ed90 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.138+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730035040 con 0x7f473815ed90 2024-03-28T17:59:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.138+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380b0760 con 0x7f473815ed90 2024-03-28T17:59:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.139+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47380680c0 con 0x7f473815ed90 2024-03-28T17:59:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.140+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f473004b060 con 0x7f473815ed90 2024-03-28T17:59:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.141+0000 7f4735600640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 0x7f471806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.141+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47300ac780 con 0x7f473815ed90 2024-03-28T17:59:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.141+0000 7f473d800640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 0x7f471806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.142+0000 7f473d800640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 0x7f471806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f472c009a60 tx=0x7f472c003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.145+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f473002f320 con 0x7f473815ed90 2024-03-28T17:59:12.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.390+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f473815c4d0 con 0x7f473815ed90 2024-03-28T17:59:12.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.391+0000 7f4735600640 1 -- 172.21.15.62:0/3988757236 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f473007b260 con 0x7f473815ed90 2024-03-28T17:59:12.391 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 msgr2=0x7f471806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 0x7f471806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f472c009a60 tx=0x7f472c003990 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 msgr2=0x7f4738142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f47300090b0 tx=0x7f47300046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 shutdown_connections 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f471806bbd0 0x7f471806e090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473815ed90 0x7f4738142670 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4738144740 0x7f47381f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 --2- 172.21.15.62:0/3988757236 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f47380b31f0 0x7f473815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.394+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 >> 172.21.15.62:0/3988757236 conn(0x7f4738066fb0 msgr2=0x7f4738067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.395+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 shutdown_connections 2024-03-28T17:59:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:12.395+0000 7f4740200640 1 -- 172.21.15.62:0/3988757236 wait complete. 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T17:59:12.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T17:59:12.416 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T17:59:12.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:12.416 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-03-28T17:59:12.469 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:12.469 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T17:59:12.470 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:12.471 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T17:59:12.471 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42573sec preferred_lft 42573sec 2024-03-28T17:59:12.471 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-03-28T17:59:12.471 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:12.472 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T17:59:12.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T17:59:12.472 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:12.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:12.931 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T17:59:13.005 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-03-28T17:59:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:13.008 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-03-28T17:59:13.069 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-03-28T17:59:13.069 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T17:59:13.069 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T17:59:13.070 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T17:59:13.070 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T17:59:13.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:13.070 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:13.070 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:13.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T17:59:13.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T17:59:13.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T17:59:13.071 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:13.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:13.814 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-03-28T17:59:13.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:13.864 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-03-28T17:59:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:13.909 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:13.909 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:13.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:13.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-28T17:59:13.909 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:13.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:59:14.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:59:14.077 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-28T17:59:14.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-28T17:59:14.077 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:14.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:14.459 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:14.459 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:14.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-03-28T17:59:14.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T17:59:14.459 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:14.612 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-28T17:59:14.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:14.612 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:14.654 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T17:59:14.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:14.655 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:14.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T17:59:14.884 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T17:59:14.929 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T17:59:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T17:59:14.931 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:14.932 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:14.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:14.933 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T17:59:15.025 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-28T17:59:15.025 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-28T17:59:15.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T17:59:15.096 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:15.138 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T17:59:15.139 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T17:59:15.140 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T17:59:15.141 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:15.141 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:15.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:15.142 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T17:59:15.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:2024-03-28T17:59:15.209+0000 7f36fcc42480 -1 init, newargv = 0x563ef07a9d60 newargc=15 2024-03-28T17:59:15.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[44971]: starting ceph client 2024-03-28T17:59:15.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[44971]: starting fuse 2024-03-28T17:59:15.228 INFO:teuthology.orchestra.run.smithi062.stdout:42 2024-03-28T17:59:15.228 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo stdin-killer -- python3 -c ' 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-03-28T17:59:15.228 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T17:59:15.229 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:15.230 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.0")) 2024-03-28T17:59:15.230 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-03-28T17:59:15.426 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.0.44971.asok 2024-03-28T17:59:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T17:59:15.443 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44971.asok 2024-03-28T17:59:15.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:15.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44971.asok status 2024-03-28T17:59:15.688 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T17:59:15.688 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-03-28T17:59:15.688 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "0", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "44971", 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4671, 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-03-28T17:59:15.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 4671 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 116265191 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T17:59:15.690 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 116265191 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.4671 192.168.0.1:0/116265191", 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.1:0/116265191", 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 8, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 20, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-03-28T17:59:15.691 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T17:59:15.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T17:59:15.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.924+0000 7ff802400640 1 Processor -- start 2024-03-28T17:59:15.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.924+0000 7ff802400640 1 -- start start 2024-03-28T17:59:15.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff802400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 0x7ff7fc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff802400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7fc15fa00 0x7ff7fc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff802400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc1603c0 con 0x7ff7fc15fe80 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff802400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc1606a0 con 0x7ff7fc15fa00 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff802400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b3cb0 con 0x7ff7fc152f90 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff7f2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7fc15fa00 0x7ff7fc0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff7f2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7fc15fa00 0x7ff7fc0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39788/0 (socket says 172.21.15.62:39788) 2024-03-28T17:59:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.926+0000 7ff7f2a00640 1 -- 172.21.15.62:0/713540652 learned_addr learned my addr 172.21.15.62:0/713540652 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7fb400640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 0x7ff7fc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7f2a00640 1 -- 172.21.15.62:0/713540652 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 msgr2=0x7ff7fc06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7f2a00640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 0x7ff7fc06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7f2a00640 1 -- 172.21.15.62:0/713540652 >> v1:172.21.15.81:6789/0 conn(0x7ff7fc15fe80 legacy=0x7ff7fc0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7f2a00640 1 -- 172.21.15.62:0/713540652 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc0b3f90 con 0x7ff7fc15fa00 2024-03-28T17:59:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.927+0000 7ff7fb400640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 0x7ff7fc06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.928+0000 7ff7f2a00640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fa00 0x7ff7fc0cee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009870 tx=0x7ff7e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43f9b9a8706c66c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.929+0000 7ff7faa00640 1 -- 172.21.15.62:0/713540652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4035020 con 0x7ff7fc15fa00 2024-03-28T17:59:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.929+0000 7ff7faa00640 1 -- 172.21.15.62:0/713540652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4031640 con 0x7ff7fc15fa00 2024-03-28T17:59:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.929+0000 7ff7faa00640 1 -- 172.21.15.62:0/713540652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e402d040 con 0x7ff7fc15fa00 2024-03-28T17:59:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 -- 172.21.15.62:0/713540652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fa00 msgr2=0x7ff7fc0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fa00 0x7ff7fc0cee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009870 tx=0x7ff7e402b8f0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 -- 172.21.15.62:0/713540652 shutdown_connections 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fa00 0x7ff7fc0cee80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 --2- 172.21.15.62:0/713540652 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc152f90 0x7ff7fc06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 -- 172.21.15.62:0/713540652 >> 172.21.15.62:0/713540652 conn(0x7ff7fc066fb0 msgr2=0x7ff7fc0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 -- 172.21.15.62:0/713540652 shutdown_connections 2024-03-28T17:59:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.930+0000 7ff802400640 1 -- 172.21.15.62:0/713540652 wait complete. 2024-03-28T17:59:15.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.931+0000 7ff802400640 1 Processor -- start 2024-03-28T17:59:15.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.932+0000 7ff802400640 1 -- start start 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc152f90 0x7ff7fc13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc15fa00 0x7ff7fc15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0cf540 con 0x7ff7fc15fe80 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0cf3c0 con 0x7ff7fc152f90 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff802400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0cf6c0 con 0x7ff7fc15fa00 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39802/0 (socket says 172.21.15.62:39802) 2024-03-28T17:59:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fbe00640 1 -- 172.21.15.62:0/1594513997 learned_addr learned my addr 172.21.15.62:0/1594513997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fb400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc152f90 0x7ff7fc13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7f2a00640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc15fa00 0x7ff7fc15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fbe00640 1 -- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc15fa00 msgr2=0x7ff7fc15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.933+0000 7ff7fbe00640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc15fa00 0x7ff7fc15d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7fbe00640 1 -- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc152f90 msgr2=0x7ff7fc13f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7fbe00640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc152f90 0x7ff7fc13f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7fbe00640 1 -- 172.21.15.62:0/1594513997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc09d7b0 con 0x7ff7fc15fe80 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7fbe00640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c440 tx=0x7ff7f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4011790 con 0x7ff7fc15fe80 2024-03-28T17:59:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4017070 con 0x7ff7fc15fe80 2024-03-28T17:59:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4011b70 con 0x7ff7fc15fe80 2024-03-28T17:59:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc13fa40 con 0x7ff7fc15fe80 2024-03-28T17:59:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.934+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc140000 con 0x7ff7fc15fe80 2024-03-28T17:59:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.935+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc0680c0 con 0x7ff7fc15fe80 2024-03-28T17:59:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.936+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff7f4011d10 con 0x7ff7fc15fe80 2024-03-28T17:59:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.937+0000 7ff7f3e00640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 0x7ff7d806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.937+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7f408a8f0 con 0x7ff7fc15fe80 2024-03-28T17:59:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.938+0000 7ff7fb400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 0x7ff7d806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.939+0000 7ff7fb400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 0x7ff7d806e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00f500 tx=0x7ff7ec007770 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:15.942+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff7f40592b0 con 0x7ff7fc15fe80 2024-03-28T17:59:16.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.182+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7fc0b3500 con 0x7ff7fc15fe80 2024-03-28T17:59:16.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.182+0000 7ff7f3e00640 1 -- 172.21.15.62:0/1594513997 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff7f4058bb0 con 0x7ff7fc15fe80 2024-03-28T17:59:16.184 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 msgr2=0x7ff7d806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 0x7ff7d806e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00f500 tx=0x7ff7ec007770 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 msgr2=0x7ff7fc15d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c440 tx=0x7ff7f400c900 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 shutdown_connections 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff7d806bbd0 0x7ff7d806e090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7fc15fe80 0x7ff7fc15d710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff7fc15fa00 0x7ff7fc15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 --2- 172.21.15.62:0/1594513997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc152f90 0x7ff7fc13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.185+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 >> 172.21.15.62:0/1594513997 conn(0x7ff7fc066fb0 msgr2=0x7ff7fc0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.186+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 shutdown_connections 2024-03-28T17:59:16.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:16.186+0000 7ff802400640 1 -- 172.21.15.62:0/1594513997 wait complete. 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T17:59:16.203 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-28T17:59:16.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.204 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.256 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42569sec preferred_lft 42569sec 2024-03-28T17:59:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:90:41:79:94:ec brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2c96:10ff:fe48:6677/64 scope link 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:90:41:79:94:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::890:41ff:fe79:94ec/64 scope link 2024-03-28T17:59:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.259 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T17:59:16.351 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-03-28T17:59:16.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.356 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-03-28T17:59:16.413 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-03-28T17:59:16.413 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T17:59:16.413 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T17:59:16.413 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T17:59:16.414 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T17:59:16.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T17:59:16.415 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:16.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.604 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-03-28T17:59:16.657 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T17:59:16.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.659 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-03-28T17:59:16.710 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-28T17:59:16.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.712 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:16.712 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:16.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:16.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-28T17:59:16.712 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:16.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-28T17:59:16.955 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether e2:c8:26:eb:30:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::e0c8:26ff:feeb:3008/64 scope link 2024-03-28T17:59:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:16.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:16.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether e2:c8:26:eb:30:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T17:59:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::e0c8:26ff:feeb:3008/64 scope link 2024-03-28T17:59:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:17.061 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-28T17:59:17.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:17.061 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:17.061 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:17.061 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-28T17:59:17.061 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-28T17:59:17.061 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-28T17:59:17.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-28T17:59:17.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-28T17:59:17.062 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:17.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:17.341 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T17:59:17.341 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T17:59:17.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2024-03-28T17:59:17.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2024-03-28T17:59:17.341 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T17:59:17.519 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-28T17:59:17.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:17.519 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:17.563 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-28T17:59:17.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:17.564 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:17.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T17:59:17.758 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T17:59:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T17:59:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T17:59:17.815 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:17.816 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:17.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:17.818 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T17:59:17.872 INFO:teuthology.orchestra.run.smithi062.stdout:42 2024-03-28T17:59:17.873 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-03-28T17:59:17.873 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-28T17:59:17.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T17:59:17.968 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T17:59:18.015 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:18.016 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:18.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:18.018 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T17:59:18.059 INFO:teuthology.orchestra.run.smithi062.stdout:42 2024-03-28T17:59:18.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi062.stderr:2024-03-28T17:59:18.073+0000 7f5976e0f480 -1 init, newargv = 0x556e90bbedc0 newargc=15 2024-03-28T17:59:18.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi062.stderr:ceph-fuse[45472]: starting ceph client 2024-03-28T17:59:18.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi062.stderr:ceph-fuse[45472]: starting fuse 2024-03-28T17:59:19.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T17:59:19.271 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:19.329 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T17:59:19.329 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T17:59:19.330 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:19.331 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:19.333 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:19.333 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T17:59:19.334 INFO:teuthology.orchestra.run.smithi062.stdout:/home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:19.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:19.335 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T17:59:19.392 INFO:teuthology.orchestra.run.smithi062.stdout:42 2024-03-28T17:59:19.392 INFO:teuthology.orchestra.run.smithi062.stdout:43 2024-03-28T17:59:19.392 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo stdin-killer -- python3 -c ' 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-03-28T17:59:19.393 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T17:59:19.394 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.2")) 2024-03-28T17:59:19.395 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-03-28T17:59:19.584 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.2.45472.asok 2024-03-28T17:59:19.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T17:59:19.602 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45472.asok 2024-03-28T17:59:19.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:19.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45472.asok status 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "2", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "45472", 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-03-28T17:59:19.848 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4683, 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 4683 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.2:0", 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 672036274 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.2:0", 2024-03-28T17:59:19.849 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 672036274 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.4683 192.168.0.2:0/672036274", 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.2:0/672036274", 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 8, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 20, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-03-28T17:59:19.850 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T17:59:19.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T17:59:20.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.078+0000 7ff50d200640 1 Processor -- start 2024-03-28T17:59:20.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.078+0000 7ff50d200640 1 -- start start 2024-03-28T17:59:20.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.079+0000 7ff50d200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff508071d10 0x7ff508169600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:20.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.079+0000 7ff50d200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080d24a0 con 0x7ff5080b3310 2024-03-28T17:59:20.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.079+0000 7ff50d200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080cf600 con 0x7ff508071930 2024-03-28T17:59:20.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.079+0000 7ff50d200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5080d3330 con 0x7ff508071d10 2024-03-28T17:59:20.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.080+0000 7ff505400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff5080b3310 0x7ff5080ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56876/0 (socket says 172.21.15.62:56876) 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.080+0000 7ff505400640 1 -- 172.21.15.62:0/1838790699 learned_addr learned my addr 172.21.15.62:0/1838790699 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.080+0000 7ff506800640 1 --2- 172.21.15.62:0/1838790699 >> v2:172.21.15.81:3300/0 conn(0x7ff508071d10 0x7ff508169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.080+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672379887 0 0) 0x7ff4f0001110 con 0x7ff5080b3310 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.080+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ec003520 con 0x7ff5080b3310 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.081+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685009756 0 0) 0x7ff4ec003520 con 0x7ff5080b3310 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.081+0000 7ff506800640 1 -- 172.21.15.62:0/1838790699 >> v1:172.21.15.81:6789/0 conn(0x7ff508071930 legacy=0x7ff5080ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.081+0000 7ff506800640 1 -- 172.21.15.62:0/1838790699 >> v1:172.21.15.62:6789/0 conn(0x7ff5080b3310 legacy=0x7ff5080ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.081+0000 7ff506800640 1 -- 172.21.15.62:0/1838790699 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080d2720 con 0x7ff508071d10 2024-03-28T17:59:20.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.082+0000 7ff506800640 1 --2- 172.21.15.62:0/1838790699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff508071d10 0x7ff508169600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d90 tx=0x7ff50002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34cce9421ec37f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:20.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.082+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500039070 con 0x7ff508071d10 2024-03-28T17:59:20.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50002c9d0 con 0x7ff508071d10 2024-03-28T17:59:20.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff504a00640 1 -- 172.21.15.62:0/1838790699 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0001840 con 0x7ff508071d10 2024-03-28T17:59:20.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 -- 172.21.15.62:0/1838790699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff508071d10 msgr2=0x7ff508169600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 --2- 172.21.15.62:0/1838790699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff508071d10 0x7ff508169600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d90 tx=0x7ff50002bea0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 -- 172.21.15.62:0/1838790699 shutdown_connections 2024-03-28T17:59:20.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 --2- 172.21.15.62:0/1838790699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff508071d10 0x7ff508169600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 -- 172.21.15.62:0/1838790699 >> 172.21.15.62:0/1838790699 conn(0x7ff5080d3ef0 msgr2=0x7ff5080d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:20.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 -- 172.21.15.62:0/1838790699 shutdown_connections 2024-03-28T17:59:20.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.083+0000 7ff50d200640 1 -- 172.21.15.62:0/1838790699 wait complete. 2024-03-28T17:59:20.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.085+0000 7ff50d200640 1 Processor -- start 2024-03-28T17:59:20.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.085+0000 7ff50d200640 1 -- start start 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff508071930 0x7ff508161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5080b3310 0x7ff5080aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080d2fc0 con 0x7ff508071d10 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080d2ce0 con 0x7ff5080b3310 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.086+0000 7ff50d200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b81a0 con 0x7ff508071930 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39816/0 (socket says 172.21.15.62:39816) 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 -- 172.21.15.62:0/2219410244 learned_addr learned my addr 172.21.15.62:0/2219410244 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505e00640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff508071930 0x7ff508161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:20.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff506800640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5080b3310 0x7ff5080aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:20.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 -- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff508071930 msgr2=0x7ff508161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff508071930 0x7ff508161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 -- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5080b3310 msgr2=0x7ff5080aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5080b3310 0x7ff5080aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 -- 172.21.15.62:0/2219410244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508160590 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.087+0000 7ff505400640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000c730 tx=0x7ff4f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.088+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0005ad0 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.088+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0005c70 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.088+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0005e60 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.088+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff508160870 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.088+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff508160e30 con 0x7ff508071d10 2024-03-28T17:59:20.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.089+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5080680c0 con 0x7ff508071d10 2024-03-28T17:59:20.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.090+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff4f0007270 con 0x7ff508071d10 2024-03-28T17:59:20.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.092+0000 7ff4fe000640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 0x7ff4e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:20.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.092+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4f008a900 con 0x7ff508071d10 2024-03-28T17:59:20.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.092+0000 7ff505e00640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 0x7ff4e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:20.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.093+0000 7ff505e00640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 0x7ff4e406e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5080cb670 tx=0x7ff4f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:20.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.097+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff5080d24a0 con 0x7ff508071d10 2024-03-28T17:59:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.340+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5080b9710 con 0x7ff508071d10 2024-03-28T17:59:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.341+0000 7ff4fe000640 1 -- 172.21.15.62:0/2219410244 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4f008a020 con 0x7ff508071d10 2024-03-28T17:59:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T17:59:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 msgr2=0x7ff4e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 0x7ff4e406e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5080cb670 tx=0x7ff4f8009040 comp rx=0 tx=0).stop 2024-03-28T17:59:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 msgr2=0x7ff50814e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000c730 tx=0x7ff4f000cbf0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 shutdown_connections 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff4e406bbd0 0x7ff4e406e090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5080b3310 0x7ff5080aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff508071d10 0x7ff50814e1c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 --2- 172.21.15.62:0/2219410244 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff508071930 0x7ff508161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 >> 172.21.15.62:0/2219410244 conn(0x7ff5080d3ef0 msgr2=0x7ff5080cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 shutdown_connections 2024-03-28T17:59:20.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:20.344+0000 7ff50d200640 1 -- 172.21.15.62:0/2219410244 wait complete. 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:20.362 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T17:59:20.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T17:59:20.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T17:59:20.363 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-28T17:59:20.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:20.363 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:20.419 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42569sec preferred_lft 42569sec 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-28T17:59:20.420 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:20.421 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T17:59:20.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add name ceph-brx type bridge 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr flush dev ceph-brx 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set ceph-brx up 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T17:59:20.421 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:20.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:20.859 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T17:59:20.937 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-28T17:59:20.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:20.941 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T17:59:20.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T17:59:20.992 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:21.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:21.702 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-28T17:59:21.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:21.756 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-03-28T17:59:21.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:21.805 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:21.805 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:21.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:21.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-28T17:59:21.806 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:21.939 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-28T17:59:21.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-28T17:59:21.939 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:22.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:22.205 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:22.205 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:22.205 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-03-28T17:59:22.205 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T17:59:22.205 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:22.364 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-28T17:59:22.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:22.364 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-28T17:59:22.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:22.413 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:22.504 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-03-28T17:59:22.587 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:22.633 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2024-03-28T17:59:22.633 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2024-03-28T17:59:22.633 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2024-03-28T17:59:22.633 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/bpf 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2024-03-28T17:59:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:22.635 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-4 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-5 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-6 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-7 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:22.636 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:22.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:22.638 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-03-28T17:59:22.725 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-28T17:59:22.725 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-28T17:59:22.730 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-03-28T17:59:22.806 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2024-03-28T17:59:22.849 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/bpf 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:22.850 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-4 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-5 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-6 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-7 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:22.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:22.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:22.852 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-03-28T17:59:22.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi081.stderr:2024-03-28T17:59:22.919+0000 7f1578242480 -1 init, newargv = 0x561667e7de50 newargc=15 2024-03-28T17:59:22.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi081.stderr:ceph-fuse[42774]: starting ceph client 2024-03-28T17:59:22.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi081.stderr:ceph-fuse[42774]: starting fuse 2024-03-28T17:59:22.938 INFO:teuthology.orchestra.run.smithi081.stdout:41 2024-03-28T17:59:22.938 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-03-28T17:59:22.939 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-03-28T17:59:22.940 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-03-28T17:59:22.941 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-03-28T17:59:22.941 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T17:59:22.941 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:22.941 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.1")) 2024-03-28T17:59:22.941 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-03-28T17:59:23.136 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.1.42774.asok 2024-03-28T17:59:23.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T17:59:23.152 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42774.asok 2024-03-28T17:59:23.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:23.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42774.asok status 2024-03-28T17:59:23.379 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "1", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "42774", 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-03-28T17:59:23.380 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4469, 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4469 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 69074750 2024-03-28T17:59:23.381 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 69074750 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4469 192.168.0.1:0/69074750", 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/69074750", 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 8, 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 20, 2024-03-28T17:59:23.382 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-03-28T17:59:23.383 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-03-28T17:59:23.383 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-03-28T17:59:23.383 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-28T17:59:23.394 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T17:59:23.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.615+0000 7f3316e00640 1 Processor -- start 2024-03-28T17:59:23.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.615+0000 7f3316e00640 1 -- start start 2024-03-28T17:59:23.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.617+0000 7f3316e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.617+0000 7f3316e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f3310072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.617+0000 7f3316e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100b30d0 con 0x7f33100719b0 2024-03-28T17:59:23.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.617+0000 7f3316e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100b33b0 con 0x7f33100ae370 2024-03-28T17:59:23.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.617+0000 7f3316e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100b3690 con 0x7f3310071d90 2024-03-28T17:59:23.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.618+0000 7f3307400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.618+0000 7f3307400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52190/0 (socket says 172.21.15.62:52190) 2024-03-28T17:59:23.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.618+0000 7f330fe00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f33100719b0 0x7f33100b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:45052/0 (socket says 172.21.15.62:45052) 2024-03-28T17:59:23.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.618+0000 7f3307400640 1 -- 172.21.15.62:0/2280447520 learned_addr learned my addr 172.21.15.62:0/2280447520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:23.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.618+0000 7f3314e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f3310072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.619+0000 7f3314e00640 1 -- 172.21.15.62:0/2280447520 >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 msgr2=0x7f33100ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:23.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.619+0000 7f3314e00640 1 --2- 172.21.15.62:0/2280447520 >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.619+0000 7f3314e00640 1 -- 172.21.15.62:0/2280447520 >> v1:172.21.15.81:6789/0 conn(0x7f33100719b0 legacy=0x7f33100b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.619+0000 7f3314e00640 1 -- 172.21.15.62:0/2280447520 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100b3970 con 0x7f3310071d90 2024-03-28T17:59:23.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.619+0000 7f3307400640 1 --2- 172.21.15.62:0/2280447520 >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:23.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.620+0000 7f3314e00640 1 --2- 172.21.15.62:0/2280447520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f3310072220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9768e8147317fa47 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:23.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.621+0000 7f330f400640 1 -- 172.21.15.62:0/2280447520 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308035020 con 0x7f3310071d90 2024-03-28T17:59:23.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.621+0000 7f330f400640 1 -- 172.21.15.62:0/2280447520 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308031810 con 0x7f3310071d90 2024-03-28T17:59:23.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.621+0000 7f330f400640 1 -- 172.21.15.62:0/2280447520 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308031be0 con 0x7f3310071d90 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 -- 172.21.15.62:0/2280447520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 msgr2=0x7f3310072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 --2- 172.21.15.62:0/2280447520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f3310072220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802b8b0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 -- 172.21.15.62:0/2280447520 shutdown_connections 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 --2- 172.21.15.62:0/2280447520 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f3310072220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 --2- 172.21.15.62:0/2280447520 >> v2:172.21.15.81:3300/0 conn(0x7f33100ae370 0x7f33100ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 -- 172.21.15.62:0/2280447520 >> 172.21.15.62:0/2280447520 conn(0x7f3310066fb0 msgr2=0x7f33100673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 -- 172.21.15.62:0/2280447520 shutdown_connections 2024-03-28T17:59:23.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.622+0000 7f3316e00640 1 -- 172.21.15.62:0/2280447520 wait complete. 2024-03-28T17:59:23.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.624+0000 7f3316e00640 1 Processor -- start 2024-03-28T17:59:23.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.624+0000 7f3316e00640 1 -- start start 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33100719b0 0x7f331015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f33100cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100aed10 con 0x7f33100ae370 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3310072760 con 0x7f33100719b0 2024-03-28T17:59:23.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3316e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331006ccb0 con 0x7f3310071d90 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3314e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3314e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39820/0 (socket says 172.21.15.62:39820) 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3314e00640 1 -- 172.21.15.62:0/1943967731 learned_addr learned my addr 172.21.15.62:0/1943967731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f330fe00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33100719b0 0x7f331015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.625+0000 7f3307400640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f33100cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 -- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 msgr2=0x7f33100cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f33100cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 -- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33100719b0 msgr2=0x7f331015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33100719b0 0x7f331015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 -- 172.21.15.62:0/1943967731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100cdf40 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.626+0000 7f3314e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f330802bec0 tx=0x7f33080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.627+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308035040 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.627+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330803ac80 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.627+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308046480 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.627+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33100ce220 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.627+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33100c9560 con 0x7f33100ae370 2024-03-28T17:59:23.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.628+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33100680c0 con 0x7f33100ae370 2024-03-28T17:59:23.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.629+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3308038020 con 0x7f33100ae370 2024-03-28T17:59:23.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.630+0000 7f330cc00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 0x7f32ec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:23.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.630+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33080ac9e0 con 0x7f33100ae370 2024-03-28T17:59:23.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.631+0000 7f330fe00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 0x7f32ec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:23.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.632+0000 7f330fe00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 0x7f32ec06e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3300007fd0 tx=0x7f3300007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:23.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.638+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f330807ae70 con 0x7f33100ae370 2024-03-28T17:59:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.880+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33100687d0 con 0x7f33100ae370 2024-03-28T17:59:23.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.881+0000 7f330cc00640 1 -- 172.21.15.62:0/1943967731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f330807ef10 con 0x7f33100ae370 2024-03-28T17:59:23.881 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T17:59:23.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 msgr2=0x7f32ec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 0x7f32ec06e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3300007fd0 tx=0x7f3300007e80 comp rx=0 tx=0).stop 2024-03-28T17:59:23.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 msgr2=0x7f33100cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:23.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f330802bec0 tx=0x7f33080046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 shutdown_connections 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f32ec06bbd0 0x7f32ec06e090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33100ae370 0x7f33100cac90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3310071d90 0x7f33100cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 --2- 172.21.15.62:0/1943967731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33100719b0 0x7f331015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 >> 172.21.15.62:0/1943967731 conn(0x7f3310066fb0 msgr2=0x7f3310067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 shutdown_connections 2024-03-28T17:59:23.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:23.884+0000 7f3316e00640 1 -- 172.21.15.62:0/1943967731 wait complete. 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:23.903 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T17:59:23.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T17:59:23.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T17:59:23.904 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-28T17:59:23.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:23.904 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:23.958 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42565sec preferred_lft 42565sec 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether b6:a7:c5:61:08:8f brd ff:ff:ff:ff:ff:ff 2024-03-28T17:59:23.959 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::58cc:4ff:fef0:7b89/64 scope link 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether b6:a7:c5:61:08:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::b4a7:c5ff:fe61:88f/64 scope link 2024-03-28T17:59:23.960 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:23.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:23.961 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T17:59:24.050 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-28T17:59:24.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.054 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-28T17:59:24.102 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-28T17:59:24.102 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T17:59:24.103 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T17:59:24.103 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T17:59:24.103 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T17:59:24.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T17:59:24.103 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:24.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.272 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-28T17:59:24.317 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-28T17:59:24.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.318 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-03-28T17:59:24.407 INFO:teuthology.orchestra.run.smithi081.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-28T17:59:24.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.408 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:24.408 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:24.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:24.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-28T17:59:24.408 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:24.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.543 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:24.625 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether da:9a:3b:62:90:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T17:59:24.626 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T17:59:24.626 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.626 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::d89a:3bff:fe62:90f1/64 scope link 2024-03-28T17:59:24.626 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-28T17:59:24.700 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T17:59:24.700 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether da:9a:3b:62:90:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::d89a:3bff:fe62:90f1/64 scope link 2024-03-28T17:59:24.701 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-28T17:59:24.703 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-28T17:59:24.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-28T17:59:24.704 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:24.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:24.955 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-28T17:59:24.955 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-28T17:59:24.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.1 up 2024-03-28T17:59:24.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.1 master ceph-brx 2024-03-28T17:59:24.955 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-28T17:59:25.118 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-28T17:59:25.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:25.119 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:25.164 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-28T17:59:25.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T17:59:25.164 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:25.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-03-28T17:59:25.382 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2024-03-28T17:59:25.432 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/bpf 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:25.433 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-4 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-5 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-6 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-7 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:25.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:25.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:25.436 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-03-28T17:59:25.480 INFO:teuthology.orchestra.run.smithi081.stdout:41 2024-03-28T17:59:25.480 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-03-28T17:59:25.480 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-28T17:59:25.526 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-03-28T17:59:25.601 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/bpf 2024-03-28T17:59:25.640 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2024-03-28T17:59:25.641 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-4 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-5 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-6 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-7 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:25.642 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:25.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:25.643 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-03-28T17:59:25.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi081.stderr:2024-03-28T17:59:25.729+0000 7fdf5c24c480 -1 init, newargv = 0x56121c9b4b20 newargc=15 2024-03-28T17:59:25.729 INFO:teuthology.orchestra.run.smithi081.stdout:41 2024-03-28T17:59:25.729 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi081.stderr:ceph-fuse[43226]: starting ceph client 2024-03-28T17:59:25.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi081.stderr:ceph-fuse[43226]: starting fuse 2024-03-28T17:59:26.831 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-03-28T17:59:26.922 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T17:59:26.967 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2024-03-28T17:59:26.967 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/bpf 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2024-03-28T17:59:26.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-4 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-5 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-6 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-7 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:26.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T17:59:26.970 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:26.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:26.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T17:59:26.970 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:26.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:26.971 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-03-28T17:59:27.066 INFO:teuthology.orchestra.run.smithi081.stdout:41 2024-03-28T17:59:27.066 INFO:teuthology.orchestra.run.smithi081.stdout:43 2024-03-28T17:59:27.066 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-03-28T17:59:27.067 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.3")) 2024-03-28T17:59:27.068 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-03-28T17:59:27.243 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.3.43226.asok 2024-03-28T17:59:27.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-28T17:59:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T17:59:27.264 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43226.asok 2024-03-28T17:59:27.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.265 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43226.asok status 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "3", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "43226", 2024-03-28T17:59:27.552 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4669, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4669 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1509656016 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:27.553 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1509656016 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4669 192.168.0.2:0/1509656016", 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.2:0/1509656016", 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 8, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 20, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-03-28T17:59:27.554 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-28T17:59:27.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.571 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-03-28T17:59:27.571 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-03-28T17:59:27.634 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:27.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:27.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.739 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-03-28T17:59:27.739 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:27.795 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-03-28T17:59:27.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:27.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:27.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.903 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-03-28T17:59:27.904 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:27.959 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-03-28T17:59:27.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:27.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:27.960 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:28.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:28.040 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-03-28T17:59:28.040 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:28.087 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-03-28T17:59:28.087 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:28.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:28.088 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-28T17:59:28.224 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-28T17:59:28.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:28.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.801+0000 7fa463e00640 1 Processor -- start 2024-03-28T17:59:28.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.801+0000 7fa463e00640 1 -- start start 2024-03-28T17:59:28.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa463e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa46406d1b0 0x7fa464072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:28.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa463e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa464071ed0 con 0x7fa46415b9c0 2024-03-28T17:59:28.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa463e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa464070a40 con 0x7fa4640722b0 2024-03-28T17:59:28.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa463e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa464071330 con 0x7fa46406d1b0 2024-03-28T17:59:28.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa462a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa46415b9c0 0x7fa4640717c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54874/0 (socket says 172.21.15.62:54874) 2024-03-28T17:59:28.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.803+0000 7fa462a00640 1 -- 172.21.15.62:0/1503354971 learned_addr learned my addr 172.21.15.62:0/1503354971 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:28.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa463400640 1 --2- 172.21.15.62:0/1503354971 >> v2:172.21.15.81:3300/0 conn(0x7fa46406d1b0 0x7fa464072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:28.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa461600640 1 -- 172.21.15.62:0/1503354971 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232022537 0 0) 0x7fa464071ed0 con 0x7fa46415b9c0 2024-03-28T17:59:28.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa461600640 1 -- 172.21.15.62:0/1503354971 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa448003520 con 0x7fa46415b9c0 2024-03-28T17:59:28.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa463400640 1 -- 172.21.15.62:0/1503354971 >> v1:172.21.15.81:6789/0 conn(0x7fa4640722b0 legacy=0x7fa46406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:28.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa463400640 1 -- 172.21.15.62:0/1503354971 >> v1:172.21.15.62:6789/0 conn(0x7fa46415b9c0 legacy=0x7fa4640717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:28.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.804+0000 7fa463400640 1 -- 172.21.15.62:0/1503354971 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640b3af0 con 0x7fa46406d1b0 2024-03-28T17:59:28.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.806+0000 7fa463400640 1 --2- 172.21.15.62:0/1503354971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464072740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4640714b0 tx=0x7fa45c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a803479c41e467 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:28.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.806+0000 7fa461600640 1 -- 172.21.15.62:0/1503354971 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c039070 con 0x7fa46406d1b0 2024-03-28T17:59:28.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.806+0000 7fa461600640 1 -- 172.21.15.62:0/1503354971 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c02cc60 con 0x7fa46406d1b0 2024-03-28T17:59:28.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.806+0000 7fa461600640 1 -- 172.21.15.62:0/1503354971 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c034640 con 0x7fa46406d1b0 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 -- 172.21.15.62:0/1503354971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 msgr2=0x7fa464072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 --2- 172.21.15.62:0/1503354971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464072740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4640714b0 tx=0x7fa45c02beb0 comp rx=0 tx=0).stop 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 -- 172.21.15.62:0/1503354971 shutdown_connections 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 --2- 172.21.15.62:0/1503354971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464072740 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 -- 172.21.15.62:0/1503354971 >> 172.21.15.62:0/1503354971 conn(0x7fa4640b1730 msgr2=0x7fa4640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 -- 172.21.15.62:0/1503354971 shutdown_connections 2024-03-28T17:59:28.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.807+0000 7fa463e00640 1 -- 172.21.15.62:0/1503354971 wait complete. 2024-03-28T17:59:28.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.808+0000 7fa463e00640 1 Processor -- start 2024-03-28T17:59:28.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.809+0000 7fa463e00640 1 -- start start 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa46415b9c0 0x7fa4640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464070fc0 con 0x7fa4640722b0 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464070ce0 con 0x7fa46406d1b0 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46415bda0 con 0x7fa46415b9c0 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38194/0 (socket says 172.21.15.62:38194) 2024-03-28T17:59:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 -- 172.21.15.62:0/1936772559 learned_addr learned my addr 172.21.15.62:0/1936772559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462a00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa463400640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa46415b9c0 0x7fa4640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 -- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa46415b9c0 msgr2=0x7fa4640ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa46415b9c0 0x7fa4640ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 -- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 msgr2=0x7fa464154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.810+0000 7fa462000640 1 -- 172.21.15.62:0/1936772559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464154c80 con 0x7fa4640722b0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa462000640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c400 tx=0x7fa45000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500052d0 con 0x7fa4640722b0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500054b0 con 0x7fa4640722b0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450006a40 con 0x7fa4640722b0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4640cc530 con 0x7fa4640722b0 2024-03-28T17:59:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.811+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4640ccaf0 con 0x7fa4640722b0 2024-03-28T17:59:28.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.812+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa464073490 con 0x7fa4640722b0 2024-03-28T17:59:28.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.813+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa450006be0 con 0x7fa4640722b0 2024-03-28T17:59:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.814+0000 7fa45aa00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 0x7fa44406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.814+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa450089f30 con 0x7fa4640722b0 2024-03-28T17:59:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.815+0000 7fa462a00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 0x7fa44406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:28.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.816+0000 7fa462a00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 0x7fa44406e090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4540011e0 tx=0x7fa454009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:28.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:28.822+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa450058a10 con 0x7fa4640722b0 2024-03-28T17:59:29.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.027+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4640aea20 con 0x7fa4640722b0 2024-03-28T17:59:29.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.030+0000 7fa45aa00640 1 -- 172.21.15.62:0/1936772559 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fa450058310 con 0x7fa4640722b0 2024-03-28T17:59:29.031 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:29.031 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:29.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 msgr2=0x7fa44406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 0x7fa44406e090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4540011e0 tx=0x7fa454009040 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 msgr2=0x7fa464149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c400 tx=0x7fa45000c8c0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 shutdown_connections 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa44406bbd0 0x7fa44406e090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa46415b9c0 0x7fa4640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4640722b0 0x7fa464149ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 --2- 172.21.15.62:0/1936772559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa46406d1b0 0x7fa464154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 >> 172.21.15.62:0/1936772559 conn(0x7fa4640b1730 msgr2=0x7fa4640d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 shutdown_connections 2024-03-28T17:59:29.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.033+0000 7fa463e00640 1 -- 172.21.15.62:0/1936772559 wait complete. 2024-03-28T17:59:29.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:29.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:29.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.277+0000 7f8ab3e00640 1 Processor -- start 2024-03-28T17:59:29.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.278+0000 7f8ab3e00640 1 -- start start 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab3e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ab415bde0 0x7f8ab40cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ab40d24f0 0x7f8ab40d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab3e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab40cf5a0 con 0x7f8ab40ce360 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40ccfb0 con 0x7f8ab40d24f0 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab3e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40cf880 con 0x7f8ab415bde0 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ab40d24f0 0x7f8ab40d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ab40d24f0 0x7f8ab40d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38204/0 (socket says 172.21.15.62:38204) 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.279+0000 7f8ab2000640 1 -- 172.21.15.62:0/4188512579 learned_addr learned my addr 172.21.15.62:0/4188512579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.280+0000 7f8ab2a00640 1 --2- 172.21.15.62:0/4188512579 >> v2:172.21.15.81:3300/0 conn(0x7f8ab415bde0 0x7f8ab40cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.280+0000 7f8ab2000640 1 -- 172.21.15.62:0/4188512579 >> v2:172.21.15.81:3300/0 conn(0x7f8ab415bde0 msgr2=0x7f8ab40cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.280+0000 7f8ab2000640 1 --2- 172.21.15.62:0/4188512579 >> v2:172.21.15.81:3300/0 conn(0x7f8ab415bde0 0x7f8ab40cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.280+0000 7f8ab2000640 1 -- 172.21.15.62:0/4188512579 >> v1:172.21.15.81:6789/0 conn(0x7f8ab40ce360 legacy=0x7f8ab40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:29.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.280+0000 7f8ab2000640 1 -- 172.21.15.62:0/4188512579 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40cd200 con 0x7f8ab40d24f0 2024-03-28T17:59:29.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.281+0000 7f8ab2000640 1 --2- 172.21.15.62:0/4188512579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40d24f0 0x7f8ab40d01a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4067db0 tx=0x7f8aa002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718216ede1582779 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.281+0000 7f8ab1600640 1 -- 172.21.15.62:0/4188512579 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0035020 con 0x7f8ab40d24f0 2024-03-28T17:59:29.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.281+0000 7f8ab1600640 1 -- 172.21.15.62:0/4188512579 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0031870 con 0x7f8ab40d24f0 2024-03-28T17:59:29.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.281+0000 7f8ab1600640 1 -- 172.21.15.62:0/4188512579 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0031c40 con 0x7f8ab40d24f0 2024-03-28T17:59:29.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 -- 172.21.15.62:0/4188512579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40d24f0 msgr2=0x7f8ab40d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/4188512579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40d24f0 0x7f8ab40d01a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4067db0 tx=0x7f8aa002b910 comp rx=0 tx=0).stop 2024-03-28T17:59:29.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 -- 172.21.15.62:0/4188512579 shutdown_connections 2024-03-28T17:59:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/4188512579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40d24f0 0x7f8ab40d01a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/4188512579 >> v2:172.21.15.81:3300/0 conn(0x7f8ab415bde0 0x7f8ab40cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 -- 172.21.15.62:0/4188512579 >> 172.21.15.62:0/4188512579 conn(0x7f8ab40d19d0 msgr2=0x7f8ab40d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 -- 172.21.15.62:0/4188512579 shutdown_connections 2024-03-28T17:59:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.282+0000 7f8ab3e00640 1 -- 172.21.15.62:0/4188512579 wait complete. 2024-03-28T17:59:29.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.284+0000 7f8ab3e00640 1 Processor -- start 2024-03-28T17:59:29.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.284+0000 7f8ab3e00640 1 -- start start 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ab40d24f0 0x7f8ab40c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 0x7f8ab40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40d08e0 con 0x7f8ab40ce360 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40cd7c0 con 0x7f8ab415bde0 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab415c1c0 con 0x7f8ab40d24f0 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38216/0 (socket says 172.21.15.62:38216) 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.285+0000 7f8ab2a00640 1 -- 172.21.15.62:0/3711715008 learned_addr learned my addr 172.21.15.62:0/3711715008 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 -- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ab40d24f0 msgr2=0x7f8ab40c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ab40d24f0 0x7f8ab40c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 -- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 msgr2=0x7f8ab40c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab3400640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 0x7f8ab40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 0x7f8ab40c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 -- 172.21.15.62:0/3711715008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40c7100 con 0x7f8ab40ce360 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab3400640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 0x7f8ab40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab2a00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d5f0 tx=0x7f8aa400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40125f0 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4019070 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40129d0 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab406cb70 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.286+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab40c73a0 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.288+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab404f400 con 0x7f8ab40ce360 2024-03-28T17:59:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.289+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8aa4012b70 con 0x7f8ab40ce360 2024-03-28T17:59:29.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.289+0000 7f8aaaa00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 0x7f8a9406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.289+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aa408ae40 con 0x7f8ab40ce360 2024-03-28T17:59:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.290+0000 7f8ab2000640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 0x7f8a9406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.291+0000 7f8ab2000640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 0x7f8a9406e0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036000 tx=0x7f8aa0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.295+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8aa4059920 con 0x7f8ab40ce360 2024-03-28T17:59:29.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.493+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ab406d090 con 0x7f8ab40ce360 2024-03-28T17:59:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.494+0000 7f8aaaa00640 1 -- 172.21.15.62:0/3711715008 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8aa4059220 con 0x7f8ab40ce360 2024-03-28T17:59:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:29.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 msgr2=0x7f8a9406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 0x7f8a9406e0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036000 tx=0x7f8aa0030000 comp rx=0 tx=0).stop 2024-03-28T17:59:29.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 msgr2=0x7f8ab4158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d5f0 tx=0x7f8aa400dab0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 shutdown_connections 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8a9406bc00 0x7f8a9406e0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ab415bde0 0x7f8ab40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8ab40d24f0 0x7f8ab40c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 --2- 172.21.15.62:0/3711715008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab40ce360 0x7f8ab4158160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 >> 172.21.15.62:0/3711715008 conn(0x7f8ab40d19d0 msgr2=0x7f8ab415ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 shutdown_connections 2024-03-28T17:59:29.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.497+0000 7f8ab3e00640 1 -- 172.21.15.62:0/3711715008 wait complete. 2024-03-28T17:59:29.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.741+0000 7f13b9a00640 1 Processor -- start 2024-03-28T17:59:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.742+0000 7f13b9a00640 1 -- start start 2024-03-28T17:59:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.743+0000 7f13b9a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13b4071dd0 0x7f13b40d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.743+0000 7f13b9a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f13b40d1960 0x7f13b415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.743+0000 7f13b9a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b40d3790 con 0x7f13b40b2f40 2024-03-28T17:59:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.743+0000 7f13b9a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40b71b0 con 0x7f13b4071dd0 2024-03-28T17:59:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.743+0000 7f13b9a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40b6ef0 con 0x7f13b40d1960 2024-03-28T17:59:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f13b40b2f40 0x7f13b40716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54880/0 (socket says 172.21.15.62:54880) 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13b4071dd0 0x7f13b40d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2a00640 1 -- 172.21.15.62:0/1958721665 learned_addr learned my addr 172.21.15.62:0/1958721665 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b3400640 1 --2- 172.21.15.62:0/1958721665 >> v2:172.21.15.81:3300/0 conn(0x7f13b40d1960 0x7f13b415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b1600640 1 -- 172.21.15.62:0/1958721665 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080525284 0 0) 0x7f13b40d3790 con 0x7f13b40b2f40 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2000640 1 -- 172.21.15.62:0/1958721665 >> v2:172.21.15.81:3300/0 conn(0x7f13b40d1960 msgr2=0x7f13b415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2000640 1 --2- 172.21.15.62:0/1958721665 >> v2:172.21.15.81:3300/0 conn(0x7f13b40d1960 0x7f13b415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2000640 1 -- 172.21.15.62:0/1958721665 >> v1:172.21.15.62:6789/0 conn(0x7f13b40b2f40 legacy=0x7f13b40716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.744+0000 7f13b2000640 1 -- 172.21.15.62:0/1958721665 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40d2ef0 con 0x7f13b4071dd0 2024-03-28T17:59:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.745+0000 7f13b2000640 1 --2- 172.21.15.62:0/1958721665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b40d2850 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13b406cfc0 tx=0x7f139c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f24422cee597fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b1600640 1 -- 172.21.15.62:0/1958721665 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c039070 con 0x7f13b4071dd0 2024-03-28T17:59:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b1600640 1 -- 172.21.15.62:0/1958721665 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c02cc80 con 0x7f13b4071dd0 2024-03-28T17:59:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b1600640 1 -- 172.21.15.62:0/1958721665 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c034660 con 0x7f13b4071dd0 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 -- 172.21.15.62:0/1958721665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 msgr2=0x7f13b40d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 --2- 172.21.15.62:0/1958721665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b40d2850 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13b406cfc0 tx=0x7f139c02bed0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 -- 172.21.15.62:0/1958721665 shutdown_connections 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 --2- 172.21.15.62:0/1958721665 >> v2:172.21.15.81:3300/0 conn(0x7f13b40d1960 0x7f13b415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 --2- 172.21.15.62:0/1958721665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b40d2850 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.746+0000 7f13b9a00640 1 -- 172.21.15.62:0/1958721665 >> 172.21.15.62:0/1958721665 conn(0x7f13b40d3d90 msgr2=0x7f13b40d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.747+0000 7f13b9a00640 1 -- 172.21.15.62:0/1958721665 shutdown_connections 2024-03-28T17:59:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.747+0000 7f13b9a00640 1 -- 172.21.15.62:0/1958721665 wait complete. 2024-03-28T17:59:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.748+0000 7f13b9a00640 1 Processor -- start 2024-03-28T17:59:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.748+0000 7f13b9a00640 1 -- start start 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13b40b2f40 0x7f13b40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13b40d1960 0x7f13b415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40d4bc0 con 0x7f13b4071dd0 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40d34b0 con 0x7f13b40d1960 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.749+0000 7f13b9a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40b3320 con 0x7f13b40b2f40 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38234/0 (socket says 172.21.15.62:38234) 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 -- 172.21.15.62:0/465281787 learned_addr learned my addr 172.21.15.62:0/465281787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2000640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13b40b2f40 0x7f13b40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 -- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13b40b2f40 msgr2=0x7f13b40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13b40b2f40 0x7f13b40cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 -- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13b40d1960 msgr2=0x7f13b415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b3400640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13b40d1960 0x7f13b415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13b40d1960 0x7f13b415d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 -- 172.21.15.62:0/465281787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b415c430 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.750+0000 7f13b2a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c4d0 tx=0x7f13a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.751+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4018070 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.751+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4007500 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.751+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b415c6b0 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.751+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004a20 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.751+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b40cb000 con 0x7f13b4071dd0 2024-03-28T17:59:29.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.752+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b415cbd0 con 0x7f13b4071dd0 2024-03-28T17:59:29.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.753+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f13a401b070 con 0x7f13b4071dd0 2024-03-28T17:59:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.754+0000 7f13aaa00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 0x7f139006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.755+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13a4089e10 con 0x7f13b4071dd0 2024-03-28T17:59:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.755+0000 7f13b2000640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 0x7f139006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.756+0000 7f13b2000640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 0x7f139006e0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f139c036000 tx=0x7f139c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:29.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.759+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f13b415cbd0 con 0x7f13b4071dd0 2024-03-28T17:59:29.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.958+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13b40680c0 con 0x7f13b4071dd0 2024-03-28T17:59:29.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.958+0000 7f13aaa00640 1 -- 172.21.15.62:0/465281787 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13a408d030 con 0x7f13b4071dd0 2024-03-28T17:59:29.959 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.961+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 msgr2=0x7f139006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.961+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 0x7f139006e0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f139c036000 tx=0x7f139c030000 comp rx=0 tx=0).stop 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.961+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 msgr2=0x7f13b41521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.961+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c4d0 tx=0x7f13a400c990 comp rx=0 tx=0).stop 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 shutdown_connections 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f139006bc00 0x7f139006e0c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13b40d1960 0x7f13b415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f13b40b2f40 0x7f13b40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 --2- 172.21.15.62:0/465281787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b4071dd0 0x7f13b41521a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 >> 172.21.15.62:0/465281787 conn(0x7f13b40d3d90 msgr2=0x7f13b40d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 shutdown_connections 2024-03-28T17:59:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:29.962+0000 7f13b9a00640 1 -- 172.21.15.62:0/465281787 wait complete. 2024-03-28T17:59:29.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:30.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.209+0000 7f6376000640 1 Processor -- start 2024-03-28T17:59:30.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.209+0000 7f6376000640 1 -- start start 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6376000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f637015e2c0 0x7f63700ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6376000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6370161bb0 con 0x7f63700b3210 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6376000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6370161e90 con 0x7f63700cec50 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6376000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6370162170 con 0x7f637015e2c0 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6375600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f63700b3210 0x7f63700d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54888/0 (socket says 172.21.15.62:54888) 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f6375600640 1 -- 172.21.15.62:0/4011200209 learned_addr learned my addr 172.21.15.62:0/4011200209 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.211+0000 7f636fe00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f637015e2c0 0x7f63700ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287841134 0 0) 0x7f6370161bb0 con 0x7f63700b3210 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6358003520 con 0x7f63700b3210 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544543103 0 0) 0x7f6358003520 con 0x7f63700b3210 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f637015e2c0 msgr2=0x7f63700ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f637015e2c0 0x7f63700ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 >> v1:172.21.15.81:6789/0 conn(0x7f63700cec50 legacy=0x7f63700cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370162450 con 0x7f63700b3210 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f63680019f0 con 0x7f63700b3210 2024-03-28T17:59:30.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 >> v1:172.21.15.62:6789/0 conn(0x7f63700b3210 legacy=0x7f63700d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63580039e0 0x7f6358003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6358004800 0x7f6358004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6358005180 0x7f635800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370162170 con 0x7f63580039e0 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635800bf00 con 0x7f6358005180 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635800c080 con 0x7f6358004800 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.212+0000 7f6374c00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63580039e0 0x7f6358003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f636fe00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6358004800 0x7f6358004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6375600640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6358005180 0x7f635800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6374c00640 1 -- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6358004800 msgr2=0x7f6358004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6374c00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6358004800 0x7f6358004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6374c00640 1 -- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6358005180 msgr2=0x7f635800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6374c00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6358005180 0x7f635800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6374c00640 1 -- 172.21.15.62:0/4011200209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358003520 con 0x7f63580039e0 2024-03-28T17:59:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f636fe00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6358004800 0x7f6358004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.213+0000 7f6375600640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6358005180 0x7f635800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.214+0000 7f6374c00640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63580039e0 0x7f6358003e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6364001750 tx=0x7f6358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b965564673fd72 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.214+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364001370 con 0x7f63580039e0 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.214+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63640052d0 con 0x7f63580039e0 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.214+0000 7f636f400640 1 -- 172.21.15.62:0/4011200209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364003830 con 0x7f63580039e0 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 -- 172.21.15.62:0/4011200209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63580039e0 msgr2=0x7f6358003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 --2- 172.21.15.62:0/4011200209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63580039e0 0x7f6358003e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6364001750 tx=0x7f6358005830 comp rx=0 tx=0).stop 2024-03-28T17:59:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6375600640 1 -- 172.21.15.62:0/4011200209 reap_dead start 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 -- 172.21.15.62:0/4011200209 shutdown_connections 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 -- 172.21.15.62:0/4011200209 >> 172.21.15.62:0/4011200209 conn(0x7f6370066fb0 msgr2=0x7f63700673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 -- 172.21.15.62:0/4011200209 shutdown_connections 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.215+0000 7f6376000640 1 -- 172.21.15.62:0/4011200209 wait complete. 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.217+0000 7f6376000640 1 Processor -- start 2024-03-28T17:59:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.217+0000 7f6376000640 1 -- start start 2024-03-28T17:59:30.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63580039e0 0x7f6370151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63701539a0 0x7f63700ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700ce570 con 0x7f6370153e20 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370162a10 con 0x7f63701539a0 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6376000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63700ce6f0 con 0x7f63580039e0 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6375600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6375600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38250/0 (socket says 172.21.15.62:38250) 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.218+0000 7f6375600640 1 -- 172.21.15.62:0/3852387890 learned_addr learned my addr 172.21.15.62:0/3852387890 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6374c00640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63580039e0 0x7f6370151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f636fe00640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63701539a0 0x7f63700ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 -- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63580039e0 msgr2=0x7f6370151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63580039e0 0x7f6370151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 -- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63701539a0 msgr2=0x7f63700ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63701539a0 0x7f63700ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 -- 172.21.15.62:0/3852387890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63701543f0 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.219+0000 7f6375600640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63700b67a0 tx=0x7f636800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.220+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800eb90 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.220+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800ed30 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.220+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63701546d0 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.220+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368014630 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.220+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6370154c90 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.221+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637004f400 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.222+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f63680147d0 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.223+0000 7f636cc00640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 0x7f635006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.223+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f636808a7f0 con 0x7f6370153e20 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.223+0000 7f6374c00640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 0x7f635006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.224+0000 7f6374c00640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 0x7f635006e0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6364033000 tx=0x7f6364009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.227+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f63680592e0 con 0x7f6370153e20 2024-03-28T17:59:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.428+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63700d2ff0 con 0x7f6370153e20 2024-03-28T17:59:30.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.429+0000 7f636cc00640 1 -- 172.21.15.62:0/3852387890 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6368058be0 con 0x7f6370153e20 2024-03-28T17:59:30.429 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 msgr2=0x7f635006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 0x7f635006e0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6364033000 tx=0x7f6364009030 comp rx=0 tx=0).stop 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 msgr2=0x7f63700cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63700b67a0 tx=0x7f636800e530 comp rx=0 tx=0).stop 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 shutdown_connections 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f635006bc00 0x7f635006e0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6370153e20 0x7f63700cd060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63701539a0 0x7f63700ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 --2- 172.21.15.62:0/3852387890 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63580039e0 0x7f6370151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 >> 172.21.15.62:0/3852387890 conn(0x7f6370066fb0 msgr2=0x7f637015bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:30.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 shutdown_connections 2024-03-28T17:59:30.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.432+0000 7f6376000640 1 -- 172.21.15.62:0/3852387890 wait complete. 2024-03-28T17:59:30.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:30.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.667+0000 7f396a200640 1 Processor -- start 2024-03-28T17:59:30.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.667+0000 7f396a200640 1 -- start start 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f396a200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3964152f90 0x7f396406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f396a200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39640cf590 0x7f39640b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f396a200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39641603c0 con 0x7f396415fa00 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f396a200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39641606a0 con 0x7f39640cf590 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f396a200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640b3cb0 con 0x7f3964152f90 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f3962a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f396415fa00 0x7f39640cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54894/0 (socket says 172.21.15.62:54894) 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f3962a00640 1 -- 172.21.15.62:0/451842159 learned_addr learned my addr 172.21.15.62:0/451842159 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f3963e00640 1 --2- 172.21.15.62:0/451842159 >> v2:172.21.15.62:3300/0 conn(0x7f39640cf590 0x7f39640b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.669+0000 7f3963400640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3964152f90 0x7f396406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3962000640 1 -- 172.21.15.62:0/451842159 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198336975 0 0) 0x7f39641603c0 con 0x7f396415fa00 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3962000640 1 -- 172.21.15.62:0/451842159 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3948003520 con 0x7f396415fa00 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3963e00640 1 -- 172.21.15.62:0/451842159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3964152f90 msgr2=0x7f396406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3963e00640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3964152f90 0x7f396406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3963e00640 1 -- 172.21.15.62:0/451842159 >> v1:172.21.15.62:6789/0 conn(0x7f396415fa00 legacy=0x7f39640cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.670+0000 7f3963e00640 1 -- 172.21.15.62:0/451842159 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39640b3f90 con 0x7f39640cf590 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.671+0000 7f3963e00640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39640cf590 0x7f39640b44b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f395c0097a0 tx=0x7f395c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ee84996708e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.671+0000 7f3962000640 1 -- 172.21.15.62:0/451842159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c035020 con 0x7f39640cf590 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.671+0000 7f3962000640 1 -- 172.21.15.62:0/451842159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c031830 con 0x7f39640cf590 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.671+0000 7f3962000640 1 -- 172.21.15.62:0/451842159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c031be0 con 0x7f39640cf590 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 -- 172.21.15.62:0/451842159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39640cf590 msgr2=0x7f39640b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39640cf590 0x7f39640b44b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f395c0097a0 tx=0x7f395c02b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 -- 172.21.15.62:0/451842159 shutdown_connections 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39640cf590 0x7f39640b44b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 --2- 172.21.15.62:0/451842159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3964152f90 0x7f396406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 -- 172.21.15.62:0/451842159 >> 172.21.15.62:0/451842159 conn(0x7f3964066fb0 msgr2=0x7f39640673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 -- 172.21.15.62:0/451842159 shutdown_connections 2024-03-28T17:59:30.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.672+0000 7f396a200640 1 -- 172.21.15.62:0/451842159 wait complete. 2024-03-28T17:59:30.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.674+0000 7f396a200640 1 Processor -- start 2024-03-28T17:59:30.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.674+0000 7f396a200640 1 -- start start 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f39640cf590 0x7f3964151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3964152f90 0x7f39640c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964160000 con 0x7f396415fa00 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396415fe80 con 0x7f3964152f90 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f396a200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3964160180 con 0x7f39640cf590 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.675+0000 7f3963e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38274/0 (socket says 172.21.15.62:38274) 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 -- 172.21.15.62:0/1189748736 learned_addr learned my addr 172.21.15.62:0/1189748736 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963400640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f39640cf590 0x7f3964151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3962a00640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3964152f90 0x7f39640c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 -- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f39640cf590 msgr2=0x7f3964151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f39640cf590 0x7f3964151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 -- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3964152f90 msgr2=0x7f39640c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3964152f90 0x7f39640c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 -- 172.21.15.62:0/1189748736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39640c9f20 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.676+0000 7f3963e00640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f395c02b910 tx=0x7f395c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.677+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c035040 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.677+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c03b840 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.677+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c03bc10 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.677+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39640c9160 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.677+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39640b0390 con 0x7f396415fa00 2024-03-28T17:59:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.678+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39640680c0 con 0x7f396415fa00 2024-03-28T17:59:30.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.679+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f395c038020 con 0x7f396415fa00 2024-03-28T17:59:30.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.680+0000 7f395b400640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 0x7f394006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.680+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f395c0acaf0 con 0x7f396415fa00 2024-03-28T17:59:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.680+0000 7f3963400640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 0x7f394006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.681+0000 7f3963400640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 0x7f394006e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39540074a0 tx=0x7f395400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:30.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.685+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f39640c9f20 con 0x7f396415fa00 2024-03-28T17:59:30.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.887+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39640b3500 con 0x7f396415fa00 2024-03-28T17:59:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.889+0000 7f395b400640 1 -- 172.21.15.62:0/1189748736 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f395c07b5d0 con 0x7f396415fa00 2024-03-28T17:59:30.890 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:30.890 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 msgr2=0x7f394006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 0x7f394006e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39540074a0 tx=0x7f395400b040 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 msgr2=0x7f39640cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f395c02b910 tx=0x7f395c0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 shutdown_connections 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f394006bbd0 0x7f394006e090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f396415fa00 0x7f39640cb530 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3964152f90 0x7f39640c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 --2- 172.21.15.62:0/1189748736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f39640cf590 0x7f3964151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 >> 172.21.15.62:0/1189748736 conn(0x7f3964066fb0 msgr2=0x7f396415e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.892+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 shutdown_connections 2024-03-28T17:59:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:30.893+0000 7f396a200640 1 -- 172.21.15.62:0/1189748736 wait complete. 2024-03-28T17:59:30.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:30.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.141+0000 7f03d9400640 1 Processor -- start 2024-03-28T17:59:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.141+0000 7f03d9400640 1 -- start start 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.142+0000 7f03d9400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f03d40cec50 0x7f03d40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.142+0000 7f03d9400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03d415e2c0 0x7f03d40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.142+0000 7f03d9400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40cf570 con 0x7f03d40b3210 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d9400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d4161bb0 con 0x7f03d40cec50 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d9400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4161e90 con 0x7f03d415e2c0 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d8a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f03d40b3210 0x7f03d40b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54900/0 (socket says 172.21.15.62:54900) 2024-03-28T17:59:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d8a00640 1 -- 172.21.15.62:0/2456216643 learned_addr learned my addr 172.21.15.62:0/2456216643 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d3e00640 1 --2- 172.21.15.62:0/2456216643 >> v2:172.21.15.81:3300/0 conn(0x7f03d40cec50 0x7f03d40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d3400640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03d415e2c0 0x7f03d40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520608737 0 0) 0x7f03d40cf570 con 0x7f03d40b3210 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.143+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b8003520 con 0x7f03d40b3210 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468122256 0 0) 0x7f03b8003520 con 0x7f03d40b3210 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03d415e2c0 msgr2=0x7f03d40cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:31.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03d415e2c0 0x7f03d40cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 >> v2:172.21.15.81:3300/0 conn(0x7f03d40cec50 msgr2=0x7f03d40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 --2- 172.21.15.62:0/2456216643 >> v2:172.21.15.81:3300/0 conn(0x7f03d40cec50 0x7f03d40ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4162170 con 0x7f03d40b3210 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f03cc001670 con 0x7f03d40b3210 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 >> v1:172.21.15.62:6789/0 conn(0x7f03d40b3210 legacy=0x7f03d40b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80039b0 0x7f03b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b8004800 0x7f03b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b8005180 0x7f03b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4161bb0 con 0x7f03b80039b0 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4161e90 con 0x7f03b8005180 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b8004600 con 0x7f03b8004800 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.144+0000 7f03d3e00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80039b0 0x7f03b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.145+0000 7f03d3e00640 1 -- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b8004800 msgr2=0x7f03b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:31.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.145+0000 7f03d3e00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b8004800 0x7f03b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.145+0000 7f03d3e00640 1 -- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b8005180 msgr2=0x7f03b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:31.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.145+0000 7f03d3e00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b8005180 0x7f03b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.145+0000 7f03d3e00640 1 -- 172.21.15.62:0/2456216643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8003520 con 0x7f03b80039b0 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.146+0000 7f03d3e00640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80039b0 0x7f03b8003dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ac0 tx=0x7f03b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432c63afa1d8eabb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.146+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc039070 con 0x7f03b80039b0 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.146+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc02c710 con 0x7f03b80039b0 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.146+0000 7f03d2a00640 1 -- 172.21.15.62:0/2456216643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009690 con 0x7f03b80039b0 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 -- 172.21.15.62:0/2456216643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80039b0 msgr2=0x7f03b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 --2- 172.21.15.62:0/2456216643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80039b0 0x7f03b8003dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ac0 tx=0x7f03b8005830 comp rx=0 tx=0).stop 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d8a00640 1 -- 172.21.15.62:0/2456216643 reap_dead start 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 -- 172.21.15.62:0/2456216643 shutdown_connections 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 -- 172.21.15.62:0/2456216643 >> 172.21.15.62:0/2456216643 conn(0x7f03d4066fb0 msgr2=0x7f03d40673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 -- 172.21.15.62:0/2456216643 shutdown_connections 2024-03-28T17:59:31.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.147+0000 7f03d9400640 1 -- 172.21.15.62:0/2456216643 wait complete. 2024-03-28T17:59:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.148+0000 7f03d9400640 1 Processor -- start 2024-03-28T17:59:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.149+0000 7f03d9400640 1 -- start start 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.149+0000 7f03d9400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b80039b0 0x7f03d40b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d9400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4155ee0 0x7f03d4156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d9400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4162a10 con 0x7f03d4156ce0 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d9400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4162730 con 0x7f03d4155ee0 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d9400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40d2fc0 con 0x7f03b80039b0 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38298/0 (socket says 172.21.15.62:38298) 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 -- 172.21.15.62:0/770441291 learned_addr learned my addr 172.21.15.62:0/770441291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 -- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b80039b0 msgr2=0x7f03d40b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b80039b0 0x7f03d40b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 -- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4155ee0 msgr2=0x7f03d4156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4155ee0 0x7f03d4156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.150+0000 7f03d3400640 1 -- 172.21.15.62:0/770441291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4153130 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03d3400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ef10 tx=0x7f03c400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4018020 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4014400 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40147d0 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d41568a0 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.151+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d41536b0 con 0x7f03d4156ce0 2024-03-28T17:59:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.152+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d404f400 con 0x7f03d4156ce0 2024-03-28T17:59:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.153+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f03c4005d10 con 0x7f03d4156ce0 2024-03-28T17:59:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.154+0000 7f03cbe00640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 0x7f03b006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.155+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03c408b4a0 con 0x7f03d4156ce0 2024-03-28T17:59:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.155+0000 7f03d3e00640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 0x7f03b006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.156+0000 7f03d3e00640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 0x7f03b006e0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036000 tx=0x7f03bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.160+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f03c4022030 con 0x7f03d4156ce0 2024-03-28T17:59:31.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.363+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03d40cfa50 con 0x7f03d4156ce0 2024-03-28T17:59:31.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.364+0000 7f03cbe00640 1 -- 172.21.15.62:0/770441291 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03c4059f80 con 0x7f03d4156ce0 2024-03-28T17:59:31.364 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:31.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 msgr2=0x7f03b006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 0x7f03b006e0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036000 tx=0x7f03bc02e000 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 msgr2=0x7f03d4157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ef10 tx=0x7f03c400b480 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 shutdown_connections 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f03b006bc00 0x7f03b006e0c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4155ee0 0x7f03d4156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d4156ce0 0x7f03d4157140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 --2- 172.21.15.62:0/770441291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03b80039b0 0x7f03d40b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 >> 172.21.15.62:0/770441291 conn(0x7f03d4066fb0 msgr2=0x7f03d40d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.366+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 shutdown_connections 2024-03-28T17:59:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.367+0000 7f03d9400640 1 -- 172.21.15.62:0/770441291 wait complete. 2024-03-28T17:59:31.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.617+0000 7f4ad4c00640 1 Processor -- start 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.617+0000 7f4ad4c00640 1 -- start start 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad015e030 0x7f4ad0161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ad4c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ad0071b90 0x7f4ad0161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ad4c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00b3470 con 0x7f4ad0072ad0 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ad4c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad01629b0 con 0x7f4ad0071b90 2024-03-28T17:59:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00b25b0 con 0x7f4ad015e030 2024-03-28T17:59:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ace800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ad0071b90 0x7f4ad0161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ace800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ad0071b90 0x7f4ad0161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38310/0 (socket says 172.21.15.62:38310) 2024-03-28T17:59:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.619+0000 7f4ace800640 1 -- 172.21.15.62:0/2329274154 learned_addr learned my addr 172.21.15.62:0/2329274154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.620+0000 7f4acde00640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad015e030 0x7f4ad0161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.620+0000 7f4ace800640 1 -- 172.21.15.62:0/2329274154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad015e030 msgr2=0x7f4ad0161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.620+0000 7f4ace800640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad015e030 0x7f4ad0161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.620+0000 7f4ace800640 1 -- 172.21.15.62:0/2329274154 >> v1:172.21.15.81:6789/0 conn(0x7f4ad0072ad0 legacy=0x7f4ad00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.620+0000 7f4ace800640 1 -- 172.21.15.62:0/2329274154 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad01626d0 con 0x7f4ad0071b90 2024-03-28T17:59:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.621+0000 7f4ace800640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0071b90 0x7f4ad0161f40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009e50 tx=0x7f4ac802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304681ea08dd0075 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.622+0000 7f4acca00640 1 -- 172.21.15.62:0/2329274154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac802caf0 con 0x7f4ad0071b90 2024-03-28T17:59:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.622+0000 7f4acca00640 1 -- 172.21.15.62:0/2329274154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac802cc90 con 0x7f4ad0071b90 2024-03-28T17:59:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.622+0000 7f4acca00640 1 -- 172.21.15.62:0/2329274154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8034610 con 0x7f4ad0071b90 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.622+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2329274154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0071b90 msgr2=0x7f4ad0161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.622+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0071b90 0x7f4ad0161f40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009e50 tx=0x7f4ac802bf50 comp rx=0 tx=0).stop 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2329274154 shutdown_connections 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0071b90 0x7f4ad0161f40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2329274154 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad015e030 0x7f4ad0161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2329274154 >> 172.21.15.62:0/2329274154 conn(0x7f4ad00b1650 msgr2=0x7f4ad00b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2329274154 shutdown_connections 2024-03-28T17:59:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.623+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2329274154 wait complete. 2024-03-28T17:59:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.625+0000 7f4ad4c00640 1 Processor -- start 2024-03-28T17:59:31.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.625+0000 7f4ad4c00640 1 -- start start 2024-03-28T17:59:31.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad0071b90 0x7f4ad0160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad015e030 0x7f4ad014b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00b37a0 con 0x7f4ad0072ad0 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0162480 con 0x7f4ad015e030 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4ad4c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00723f0 con 0x7f4ad0071b90 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.626+0000 7f4acd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38324/0 (socket says 172.21.15.62:38324) 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 -- 172.21.15.62:0/2796184559 learned_addr learned my addr 172.21.15.62:0/2796184559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acde00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad0071b90 0x7f4ad0160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4ace800640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad015e030 0x7f4ad014b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 -- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad0071b90 msgr2=0x7f4ad0160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad0071b90 0x7f4ad0160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 -- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad015e030 msgr2=0x7f4ad014b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad015e030 0x7f4ad014b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 -- 172.21.15.62:0/2796184559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad015af80 con 0x7f4ad0072ad0 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.627+0000 7f4acd400640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c440 tx=0x7f4abc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.628+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0052d0 con 0x7f4ad0072ad0 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.628+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc005470 con 0x7f4ad0072ad0 2024-03-28T17:59:31.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.628+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc007260 con 0x7f4ad0072ad0 2024-03-28T17:59:31.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.628+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad015b200 con 0x7f4ad0072ad0 2024-03-28T17:59:31.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.628+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad015b7c0 con 0x7f4ad0072ad0 2024-03-28T17:59:31.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.629+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad00680c0 con 0x7f4ad0072ad0 2024-03-28T17:59:31.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.630+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4abc007400 con 0x7f4ad0072ad0 2024-03-28T17:59:31.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.631+0000 7f4ac6000640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 0x7f4ab006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:31.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.631+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4abc089fa0 con 0x7f4ad0072ad0 2024-03-28T17:59:31.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.632+0000 7f4acde00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 0x7f4ab006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:31.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.633+0000 7f4acde00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 0x7f4ab006e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:31.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.636+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4ad015af80 con 0x7f4ad0072ad0 2024-03-28T17:59:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.835+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ad00d0420 con 0x7f4ad0072ad0 2024-03-28T17:59:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.836+0000 7f4ac6000640 1 -- 172.21.15.62:0/2796184559 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4abc058a80 con 0x7f4ad0072ad0 2024-03-28T17:59:31.836 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 msgr2=0x7f4ab006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 0x7f4ab006e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2024-03-28T17:59:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 msgr2=0x7f4ad015aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c440 tx=0x7f4abc001e70 comp rx=0 tx=0).stop 2024-03-28T17:59:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 shutdown_connections 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4ab006bbd0 0x7f4ab006e090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ad015e030 0x7f4ad014b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0072ad0 0x7f4ad015aa40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 --2- 172.21.15.62:0/2796184559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4ad0071b90 0x7f4ad0160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 >> 172.21.15.62:0/2796184559 conn(0x7f4ad00b1650 msgr2=0x7f4ad00b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.838+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 shutdown_connections 2024-03-28T17:59:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:31.839+0000 7f4ad4c00640 1 -- 172.21.15.62:0/2796184559 wait complete. 2024-03-28T17:59:31.858 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.080+0000 7ff24de00640 1 Processor -- start 2024-03-28T17:59:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.080+0000 7ff24de00640 1 -- start start 2024-03-28T17:59:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2480b15c0 0x7ff2480712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24de00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff2480b2930 0x7ff24806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24de00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff248147310 con 0x7ff2481618b0 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24de00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24806d0d0 con 0x7ff2480b15c0 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24de00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff248162020 con 0x7ff2480b2930 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2480b15c0 0x7ff2480712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2480b15c0 0x7ff2480712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38338/0 (socket says 172.21.15.62:38338) 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 -- 172.21.15.62:0/4055913559 learned_addr learned my addr 172.21.15.62:0/4055913559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24d400640 1 --2- 172.21.15.62:0/4055913559 >> v2:172.21.15.81:3300/0 conn(0x7ff2480b2930 0x7ff24806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 -- 172.21.15.62:0/4055913559 >> v2:172.21.15.81:3300/0 conn(0x7ff2480b2930 msgr2=0x7ff24806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 --2- 172.21.15.62:0/4055913559 >> v2:172.21.15.81:3300/0 conn(0x7ff2480b2930 0x7ff24806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 -- 172.21.15.62:0/4055913559 >> v1:172.21.15.81:6789/0 conn(0x7ff2481618b0 legacy=0x7ff2480b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.082+0000 7ff24ca00640 1 -- 172.21.15.62:0/4055913559 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff248162300 con 0x7ff2480b15c0 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.084+0000 7ff24ca00640 1 --2- 172.21.15.62:0/4055913559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b15c0 0x7ff2480712a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff248072d40 tx=0x7ff23c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97093d896354113 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.084+0000 7ff247400640 1 -- 172.21.15.62:0/4055913559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c02e070 con 0x7ff2480b15c0 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.084+0000 7ff247400640 1 -- 172.21.15.62:0/4055913559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c037070 con 0x7ff2480b15c0 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.084+0000 7ff247400640 1 -- 172.21.15.62:0/4055913559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c031a00 con 0x7ff2480b15c0 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 -- 172.21.15.62:0/4055913559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b15c0 msgr2=0x7ff2480712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 --2- 172.21.15.62:0/4055913559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b15c0 0x7ff2480712a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff248072d40 tx=0x7ff23c02b880 comp rx=0 tx=0).stop 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 -- 172.21.15.62:0/4055913559 shutdown_connections 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 --2- 172.21.15.62:0/4055913559 >> v2:172.21.15.81:3300/0 conn(0x7ff2480b2930 0x7ff24806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 --2- 172.21.15.62:0/4055913559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b15c0 0x7ff2480712a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 -- 172.21.15.62:0/4055913559 >> 172.21.15.62:0/4055913559 conn(0x7ff248066dd0 msgr2=0x7ff2480671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 -- 172.21.15.62:0/4055913559 shutdown_connections 2024-03-28T17:59:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.085+0000 7ff24de00640 1 -- 172.21.15.62:0/4055913559 wait complete. 2024-03-28T17:59:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.087+0000 7ff24de00640 1 Processor -- start 2024-03-28T17:59:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.087+0000 7ff24de00640 1 -- start start 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2480b15c0 0x7ff248160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff2481618b0 0x7ff24813bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480b2d60 con 0x7ff2480b2930 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248161d40 con 0x7ff2480b15c0 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24de00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480b2ee0 con 0x7ff2481618b0 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff247e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff247e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38344/0 (socket says 172.21.15.62:38344) 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff247e00640 1 -- 172.21.15.62:0/1118790930 learned_addr learned my addr 172.21.15.62:0/1118790930 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24ca00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2480b15c0 0x7ff248160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.088+0000 7ff24d400640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff2481618b0 0x7ff24813bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 -- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff2481618b0 msgr2=0x7ff24813bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff2481618b0 0x7ff24813bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 -- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2480b15c0 msgr2=0x7ff248160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2480b15c0 0x7ff248160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 -- 172.21.15.62:0/1118790930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24813c530 con 0x7ff2480b2930 2024-03-28T17:59:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff247e00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff23400b3e0 tx=0x7ff23400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234015020 con 0x7ff2480b2930 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff234011850 con 0x7ff2480b2930 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234011c40 con 0x7ff2480b2930 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24813dac0 con 0x7ff2480b2930 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.089+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24813e080 con 0x7ff2480b2930 2024-03-28T17:59:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.091+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24804f400 con 0x7ff2480b2930 2024-03-28T17:59:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.092+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff234018020 con 0x7ff2480b2930 2024-03-28T17:59:32.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.093+0000 7ff244c00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 0x7ff22806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.093+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff23408a9b0 con 0x7ff2480b2930 2024-03-28T17:59:32.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.094+0000 7ff24ca00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 0x7ff22806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.095+0000 7ff24ca00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 0x7ff22806e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff23c036000 tx=0x7ff23c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.098+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff23401f020 con 0x7ff2480b2930 2024-03-28T17:59:32.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.302+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2480cf5f0 con 0x7ff2480b2930 2024-03-28T17:59:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.303+0000 7ff244c00640 1 -- 172.21.15.62:0/1118790930 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff24813c530 con 0x7ff2480b2930 2024-03-28T17:59:32.303 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:32.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 msgr2=0x7ff22806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 0x7ff22806e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff23c036000 tx=0x7ff23c03d040 comp rx=0 tx=0).stop 2024-03-28T17:59:32.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 msgr2=0x7ff24815b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff23400b3e0 tx=0x7ff23400b8a0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 shutdown_connections 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff22806bc00 0x7ff22806e0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff2481618b0 0x7ff24813bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2480b2930 0x7ff24815b4c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 --2- 172.21.15.62:0/1118790930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2480b15c0 0x7ff248160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 >> 172.21.15.62:0/1118790930 conn(0x7ff248066dd0 msgr2=0x7ff24806d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 shutdown_connections 2024-03-28T17:59:32.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.306+0000 7ff24de00640 1 -- 172.21.15.62:0/1118790930 wait complete. 2024-03-28T17:59:32.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.544+0000 7f1515000640 1 Processor -- start 2024-03-28T17:59:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.544+0000 7f1515000640 1 -- start start 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f1515000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f151015ecb0 0x7f1510071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f1515000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100b3700 con 0x7f1510144740 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f1515000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100b39e0 con 0x7f151015e850 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f1515000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100b3cc0 con 0x7f151015ecb0 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f150de00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1510144740 0x7f15100b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54902/0 (socket says 172.21.15.62:54902) 2024-03-28T17:59:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f150de00640 1 -- 172.21.15.62:0/4177323714 learned_addr learned my addr 172.21.15.62:0/4177323714 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.546+0000 7f150e800640 1 --2- 172.21.15.62:0/4177323714 >> v2:172.21.15.81:3300/0 conn(0x7f151015ecb0 0x7f1510071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248482325 0 0) 0x7f15100b3700 con 0x7f1510144740 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f4005730 con 0x7f1510144740 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413687018 0 0) 0x7f14f4005730 con 0x7f1510144740 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 >> v2:172.21.15.81:3300/0 conn(0x7f151015ecb0 msgr2=0x7f1510071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 --2- 172.21.15.62:0/4177323714 >> v2:172.21.15.81:3300/0 conn(0x7f151015ecb0 0x7f1510071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 >> v1:172.21.15.81:6789/0 conn(0x7f151015e850 legacy=0x7f1510071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100b3fa0 con 0x7f1510144740 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f1500001670 con 0x7f1510144740 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 >> v1:172.21.15.62:6789/0 conn(0x7f1510144740 legacy=0x7f15100b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14f4005bc0 0x7f14f4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 0x7f14f4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f40073d0 0x7f14f400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b3cc0 con 0x7f14f4006a10 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f400e270 con 0x7f14f4005bc0 2024-03-28T17:59:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f400e470 con 0x7f14f40073d0 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.547+0000 7f150d400640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 0x7f14f4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150de00640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14f4005bc0 0x7f14f4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150e800640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f40073d0 0x7f14f400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150d400640 1 -- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f40073d0 msgr2=0x7f14f400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150d400640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f40073d0 0x7f14f400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150d400640 1 -- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14f4005bc0 msgr2=0x7f14f4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150d400640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14f4005bc0 0x7f14f4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150d400640 1 -- 172.21.15.62:0/4177323714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4005730 con 0x7f14f4006a10 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150e800640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f40073d0 0x7f14f400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.548+0000 7f150e800640 1 -- 172.21.15.62:0/4177323714 reap_dead start 2024-03-28T17:59:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.549+0000 7f150d400640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 0x7f14f4006e90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f14fc001750 tx=0x7f14f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d557748eebf049a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.549+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc001370 con 0x7f14f4006a10 2024-03-28T17:59:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.549+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc0052d0 con 0x7f14f4006a10 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.549+0000 7f150ca00640 1 -- 172.21.15.62:0/4177323714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc003830 con 0x7f14f4006a10 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 -- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 msgr2=0x7f14f4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 0x7f14f4006e90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f14fc001750 tx=0x7f14f4007a40 comp rx=0 tx=0).stop 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 -- 172.21.15.62:0/4177323714 shutdown_connections 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 --2- 172.21.15.62:0/4177323714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f4006a10 0x7f14f4006e90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 -- 172.21.15.62:0/4177323714 >> 172.21.15.62:0/4177323714 conn(0x7f1510066fb0 msgr2=0x7f15100673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 -- 172.21.15.62:0/4177323714 shutdown_connections 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.550+0000 7f1515000640 1 -- 172.21.15.62:0/4177323714 wait complete. 2024-03-28T17:59:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.552+0000 7f1515000640 1 Processor -- start 2024-03-28T17:59:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.552+0000 7f1515000640 1 -- start start 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f4006a10 0x7f15101f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15101f6800 0x7f1510071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100ad0a0 con 0x7f151015c360 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100acf20 con 0x7f15101f6800 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f1515000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b2cb0 con 0x7f14f4006a10 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f150d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f150d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38374/0 (socket says 172.21.15.62:38374) 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.553+0000 7f150d400640 1 -- 172.21.15.62:0/1209262789 learned_addr learned my addr 172.21.15.62:0/1209262789 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150de00640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f4006a10 0x7f15101f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150e800640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15101f6800 0x7f1510071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 -- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f4006a10 msgr2=0x7f15101f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f4006a10 0x7f15101f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 -- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15101f6800 msgr2=0x7f1510071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15101f6800 0x7f1510071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 -- 172.21.15.62:0/1209262789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151015c7e0 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.554+0000 7f150d400640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005040 tx=0x7f14fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.555+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc041070 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.555+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc005480 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.555+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc007a30 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.555+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151015d300 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.555+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151015cd40 con 0x7f151015c360 2024-03-28T17:59:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.556+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15100680c0 con 0x7f151015c360 2024-03-28T17:59:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.557+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f14fc044070 con 0x7f151015c360 2024-03-28T17:59:32.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.558+0000 7f1506000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 0x7f14f006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:32.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.558+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14fc0ac520 con 0x7f151015c360 2024-03-28T17:59:32.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.558+0000 7f150de00640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 0x7f14f006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.559+0000 7f150de00640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 0x7f14f006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1500001050 tx=0x7f1500009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:32.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.562+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f151015c7e0 con 0x7f151015c360 2024-03-28T17:59:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.765+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15100ca220 con 0x7f151015c360 2024-03-28T17:59:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.769+0000 7f1506000640 1 -- 172.21.15.62:0/1209262789 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f14fc07b000 con 0x7f151015c360 2024-03-28T17:59:32.769 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:32.769 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.771+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 msgr2=0x7f14f006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.771+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 0x7f14f006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1500001050 tx=0x7f1500009040 comp rx=0 tx=0).stop 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.771+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 msgr2=0x7f15101f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.771+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005040 tx=0x7f14fc001120 comp rx=0 tx=0).stop 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 shutdown_connections 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f14f006bbd0 0x7f14f006e090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15101f6800 0x7f1510071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f151015c360 0x7f15101f4640 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 --2- 172.21.15.62:0/1209262789 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f14f4006a10 0x7f15101f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:32.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 >> 172.21.15.62:0/1209262789 conn(0x7f1510066fb0 msgr2=0x7f15100b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:32.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 shutdown_connections 2024-03-28T17:59:32.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:32.772+0000 7f1515000640 1 -- 172.21.15.62:0/1209262789 wait complete. 2024-03-28T17:59:32.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:32.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.018+0000 7f9c71800640 1 Processor -- start 2024-03-28T17:59:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.018+0000 7f9c71800640 1 -- start start 2024-03-28T17:59:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c71800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c158550 0x7f9c6c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c71800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c71800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6c15be10 con 0x7f9c6c06e040 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c71800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c06e480 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c71800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c158930 con 0x7f9c6c158550 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c70e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c70e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38376/0 (socket says 172.21.15.62:38376) 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.020+0000 7f9c70e00640 1 -- 172.21.15.62:0/2842270098 learned_addr learned my addr 172.21.15.62:0/2842270098 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.021+0000 7f9c6be00640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c158550 0x7f9c6c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.021+0000 7f9c70e00640 1 -- 172.21.15.62:0/2842270098 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c158550 msgr2=0x7f9c6c06db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.021+0000 7f9c70e00640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c158550 0x7f9c6c06db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.021+0000 7f9c70e00640 1 -- 172.21.15.62:0/2842270098 >> v1:172.21.15.81:6789/0 conn(0x7f9c6c06e040 legacy=0x7f9c6c15e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.021+0000 7f9c70e00640 1 -- 172.21.15.62:0/2842270098 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c15b050 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.022+0000 7f9c70e00640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400bb10 tx=0x7f9c6402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66416c7d896f45e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.022+0000 7f9c6aa00640 1 -- 172.21.15.62:0/2842270098 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6403b070 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.022+0000 7f9c6aa00640 1 -- 172.21.15.62:0/2842270098 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c64036440 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.022+0000 7f9c6aa00640 1 -- 172.21.15.62:0/2842270098 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64036810 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 -- 172.21.15.62:0/2842270098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c15ecb0 msgr2=0x7f9c6c0713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400bb10 tx=0x7f9c6402e0c0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 -- 172.21.15.62:0/2842270098 shutdown_connections 2024-03-28T17:59:33.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c15ecb0 0x7f9c6c0713f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 --2- 172.21.15.62:0/2842270098 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c158550 0x7f9c6c06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 -- 172.21.15.62:0/2842270098 >> 172.21.15.62:0/2842270098 conn(0x7f9c6c066dd0 msgr2=0x7f9c6c0671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:33.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 -- 172.21.15.62:0/2842270098 shutdown_connections 2024-03-28T17:59:33.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.023+0000 7f9c71800640 1 -- 172.21.15.62:0/2842270098 wait complete. 2024-03-28T17:59:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.025+0000 7f9c71800640 1 Processor -- start 2024-03-28T17:59:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.025+0000 7f9c71800640 1 -- start start 2024-03-28T17:59:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c6c158550 0x7f9c6c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c15ecb0 0x7f9c6c1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c15b8f0 con 0x7f9c6c06e040 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c15b610 con 0x7f9c6c158550 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c71800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c15f310 con 0x7f9c6c15ecb0 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c6be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38388/0 (socket says 172.21.15.62:38388) 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c6be00640 1 -- 172.21.15.62:0/3363817706 learned_addr learned my addr 172.21.15.62:0/3363817706 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c70e00640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c15ecb0 0x7f9c6c1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.026+0000 7f9c6be00640 1 -- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c15ecb0 msgr2=0x7f9c6c1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c6be00640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c15ecb0 0x7f9c6c1f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c6be00640 1 -- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c6c158550 msgr2=0x7f9c6c14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c6be00640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c6c158550 0x7f9c6c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c6be00640 1 -- 172.21.15.62:0/3363817706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c1596d0 con 0x7f9c6c06e040 2024-03-28T17:59:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c6be00640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c007ef0 tx=0x7f9c5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c014620 con 0x7f9c6c06e040 2024-03-28T17:59:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c019070 con 0x7f9c6c06e040 2024-03-28T17:59:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c159990 con 0x7f9c6c06e040 2024-03-28T17:59:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0149e0 con 0x7f9c6c06e040 2024-03-28T17:59:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.027+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c159f50 con 0x7f9c6c06e040 2024-03-28T17:59:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.028+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c04f400 con 0x7f9c6c06e040 2024-03-28T17:59:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.029+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c5c014b80 con 0x7f9c6c06e040 2024-03-28T17:59:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.030+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 0x7f9c4806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.030+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c5c08b3f0 con 0x7f9c6c06e040 2024-03-28T17:59:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.031+0000 7f9c6b400640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 0x7f9c4806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.032+0000 7f9c6b400640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 0x7f9c4806e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c1f8800 tx=0x7f9c540079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.035+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9c5c059ed0 con 0x7f9c6c06e040 2024-03-28T17:59:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.235+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c6c072b50 con 0x7f9c6c06e040 2024-03-28T17:59:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.236+0000 7f9c63e00640 1 -- 172.21.15.62:0/3363817706 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c5c0597d0 con 0x7f9c6c06e040 2024-03-28T17:59:33.236 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 msgr2=0x7f9c4806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 0x7f9c4806e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c1f8800 tx=0x7f9c540079d0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 msgr2=0x7f9c6c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c007ef0 tx=0x7f9c5c00e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 shutdown_connections 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9c4806bbd0 0x7f9c4806e090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9c6c15ecb0 0x7f9c6c1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c6c158550 0x7f9c6c14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 --2- 172.21.15.62:0/3363817706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c6c06e040 0x7f9c6c0b1f60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 >> 172.21.15.62:0/3363817706 conn(0x7f9c6c066dd0 msgr2=0x7f9c6c0ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 shutdown_connections 2024-03-28T17:59:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.238+0000 7f9c71800640 1 -- 172.21.15.62:0/3363817706 wait complete. 2024-03-28T17:59:33.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:33.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.479+0000 7f43f7e00640 1 Processor -- start 2024-03-28T17:59:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.479+0000 7f43f7e00640 1 -- start start 2024-03-28T17:59:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.481+0000 7f43f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f43f80aee00 0x7f43f80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.481+0000 7f43f7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f815bec0 con 0x7f43f8160190 2024-03-28T17:59:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.481+0000 7f43f7e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80b1bf0 con 0x7f43f80b44f0 2024-03-28T17:59:33.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.481+0000 7f43f7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80b1ed0 con 0x7f43f80aee00 2024-03-28T17:59:33.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43efe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f43f8160190 0x7f43f815ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:54914/0 (socket says 172.21.15.62:54914) 2024-03-28T17:59:33.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43efe00640 1 -- 172.21.15.62:0/3902314034 learned_addr learned my addr 172.21.15.62:0/3902314034 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6a00640 1 --2- 172.21.15.62:0/3902314034 >> v2:172.21.15.62:3300/0 conn(0x7f43f80aee00 0x7f43f80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318931420 0 0) 0x7f43f815bec0 con 0x7f43f8160190 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43dc003520 con 0x7f43f8160190 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4093746105 0 0) 0x7f43dc003520 con 0x7f43f8160190 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6a00640 1 -- 172.21.15.62:0/3902314034 >> v1:172.21.15.81:6789/0 conn(0x7f43f80b44f0 legacy=0x7f43f80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.482+0000 7f43f6a00640 1 -- 172.21.15.62:0/3902314034 >> v1:172.21.15.62:6789/0 conn(0x7f43f8160190 legacy=0x7f43f815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.483+0000 7f43f6a00640 1 -- 172.21.15.62:0/3902314034 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f815f6b0 con 0x7f43f80aee00 2024-03-28T17:59:33.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f6a00640 1 --2- 172.21.15.62:0/3902314034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80b15c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43e80097a0 tx=0x7f43e802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1b8f825a68fa5c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8038070 con 0x7f43f80aee00 2024-03-28T17:59:33.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8031500 con 0x7f43f80aee00 2024-03-28T17:59:33.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f6000640 1 -- 172.21.15.62:0/3902314034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0001670 con 0x7f43f80aee00 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f7e00640 1 -- 172.21.15.62:0/3902314034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 msgr2=0x7f43f80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.484+0000 7f43f7e00640 1 --2- 172.21.15.62:0/3902314034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80b15c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43e80097a0 tx=0x7f43e802b820 comp rx=0 tx=0).stop 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.485+0000 7f43f7e00640 1 -- 172.21.15.62:0/3902314034 shutdown_connections 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.485+0000 7f43f7e00640 1 --2- 172.21.15.62:0/3902314034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80b15c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.485+0000 7f43f7e00640 1 -- 172.21.15.62:0/3902314034 >> 172.21.15.62:0/3902314034 conn(0x7f43f8066dd0 msgr2=0x7f43f80671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.485+0000 7f43f7e00640 1 -- 172.21.15.62:0/3902314034 shutdown_connections 2024-03-28T17:59:33.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.485+0000 7f43f7e00640 1 -- 172.21.15.62:0/3902314034 wait complete. 2024-03-28T17:59:33.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.487+0000 7f43f7e00640 1 Processor -- start 2024-03-28T17:59:33.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.487+0000 7f43f7e00640 1 -- start start 2024-03-28T17:59:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43f80b44f0 0x7f43f814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43f8160190 0x7f43f8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80af1e0 con 0x7f43f80aee00 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f815fc70 con 0x7f43f8160190 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f7e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80b4930 con 0x7f43f80b44f0 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38402/0 (socket says 172.21.15.62:38402) 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 -- 172.21.15.62:0/2027973089 learned_addr learned my addr 172.21.15.62:0/2027973089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 -- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43f80b44f0 msgr2=0x7f43f814b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43f80b44f0 0x7f43f814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 -- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43f8160190 msgr2=0x7f43f8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43f8160190 0x7f43f8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.488+0000 7f43f6a00640 1 -- 172.21.15.62:0/2027973089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f815a4f0 con 0x7f43f80aee00 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43f6a00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43e80098c0 tx=0x7f43e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8043070 con 0x7f43f80aee00 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e803f870 con 0x7f43f80aee00 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8038040 con 0x7f43f80aee00 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f81577f0 con 0x7f43f80aee00 2024-03-28T17:59:33.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.489+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f8157db0 con 0x7f43f80aee00 2024-03-28T17:59:33.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.491+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f804f400 con 0x7f43f80aee00 2024-03-28T17:59:33.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.491+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f43e8046070 con 0x7f43f80aee00 2024-03-28T17:59:33.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.492+0000 7f43eea00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 0x7f43d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.492+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e80ac650 con 0x7f43f80aee00 2024-03-28T17:59:33.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.493+0000 7f43efe00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 0x7f43d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.494+0000 7f43efe00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 0x7f43d406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43f815b290 tx=0x7f43e00010a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.497+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f43f815a4f0 con 0x7f43f80aee00 2024-03-28T17:59:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.699+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43f80b3a00 con 0x7f43f80aee00 2024-03-28T17:59:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.700+0000 7f43eea00640 1 -- 172.21.15.62:0/2027973089 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43e8030070 con 0x7f43f80aee00 2024-03-28T17:59:33.700 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 msgr2=0x7f43d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 0x7f43d406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43f815b290 tx=0x7f43e00010a0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 msgr2=0x7f43f80c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43e80098c0 tx=0x7f43e80046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 shutdown_connections 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f43d406bbd0 0x7f43d406e090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43f8160190 0x7f43f8159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f43f80b44f0 0x7f43f814b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 --2- 172.21.15.62:0/2027973089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f80aee00 0x7f43f80c49a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 >> 172.21.15.62:0/2027973089 conn(0x7f43f8066dd0 msgr2=0x7f43f80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 shutdown_connections 2024-03-28T17:59:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.703+0000 7f43f7e00640 1 -- 172.21.15.62:0/2027973089 wait complete. 2024-03-28T17:59:33.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:33.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.953+0000 7f3687e00640 1 Processor -- start 2024-03-28T17:59:33.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.953+0000 7f3687e00640 1 -- start start 2024-03-28T17:59:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.954+0000 7f3687e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f36881618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3687e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3688147310 con 0x7f36880b2930 2024-03-28T17:59:33.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3687e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3688071700 con 0x7f36880b15c0 2024-03-28T17:59:33.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3687e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3688161df0 con 0x7f36880712a0 2024-03-28T17:59:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3687400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f36880b2930 0x7f368806d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41846/0 (socket says 172.21.15.62:41846) 2024-03-28T17:59:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3687400640 1 -- 172.21.15.62:0/3809385325 learned_addr learned my addr 172.21.15.62:0/3809385325 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f367fe00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f36881618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485316928 0 0) 0x7f3688147310 con 0x7f36880b2930 2024-03-28T17:59:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.955+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c003520 con 0x7f36880b2930 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313621581 0 0) 0x7f366c003520 con 0x7f36880b2930 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 msgr2=0x7f36881618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f36881618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 >> v1:172.21.15.81:6789/0 conn(0x7f36880b15c0 legacy=0x7f36880b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688162020 con 0x7f36880b2930 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f3680001a10 con 0x7f36880b2930 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 >> v1:172.21.15.62:6789/0 conn(0x7f36880b2930 legacy=0x7f368806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0039e0 0x7f366c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f366c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00baf0 con 0x7f366c0039e0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00bca0 con 0x7f366c004800 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00bea0 con 0x7f36880712a0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.956+0000 7f3686a00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0039e0 0x7f366c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f367fe00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3687400640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f366c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3686a00640 1 -- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 msgr2=0x7f366c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3686a00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f366c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3686a00640 1 -- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c004800 msgr2=0x7f366c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3686a00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3686a00640 1 -- 172.21.15.62:0/3809385325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c003520 con 0x7f366c0039e0 2024-03-28T17:59:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.957+0000 7f3687400640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f36880712a0 0x7f366c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:33.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.958+0000 7f3686a00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0039e0 0x7f366c003e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3678001750 tx=0x7f366c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24450497cd795cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.958+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678001370 con 0x7f366c0039e0 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.958+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780052d0 con 0x7f366c0039e0 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.958+0000 7f3686000640 1 -- 172.21.15.62:0/3809385325 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678003830 con 0x7f366c0039e0 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 -- 172.21.15.62:0/3809385325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0039e0 msgr2=0x7f366c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 --2- 172.21.15.62:0/3809385325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0039e0 0x7f366c003e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3678001750 tx=0x7f366c005830 comp rx=0 tx=0).stop 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687400640 1 -- 172.21.15.62:0/3809385325 reap_dead start 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 -- 172.21.15.62:0/3809385325 shutdown_connections 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 -- 172.21.15.62:0/3809385325 >> 172.21.15.62:0/3809385325 conn(0x7f3688066dd0 msgr2=0x7f36880671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 -- 172.21.15.62:0/3809385325 shutdown_connections 2024-03-28T17:59:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.959+0000 7f3687e00640 1 -- 172.21.15.62:0/3809385325 wait complete. 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.961+0000 7f3687e00640 1 Processor -- start 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.961+0000 7f3687e00640 1 -- start start 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f366c0039e0 0x7f36880c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 0x7f36881f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881625e0 con 0x7f36881f7630 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368806d0d0 con 0x7f36880b1f50 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f3687e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36880b2d80 con 0x7f366c0039e0 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f367fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f367fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46188/0 (socket says 172.21.15.62:46188) 2024-03-28T17:59:33.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.962+0000 7f367fe00640 1 -- 172.21.15.62:0/1277410028 learned_addr learned my addr 172.21.15.62:0/1277410028 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f3686a00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f366c0039e0 0x7f36880c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f3687400640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 0x7f36881f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 -- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f366c0039e0 msgr2=0x7f36880c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f366c0039e0 0x7f36880c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 -- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 msgr2=0x7f36881f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 0x7f36881f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 -- 172.21.15.62:0/1277410028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36881f7e10 con 0x7f36881f7630 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f3687400640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 0x7f36881f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.963+0000 7f367fe00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3670011630 tx=0x7f3670011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.964+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670015870 con 0x7f36881f7630 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.964+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670015a10 con 0x7f36881f7630 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.964+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670015dc0 con 0x7f36881f7630 2024-03-28T17:59:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.964+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36881f80f0 con 0x7f36881f7630 2024-03-28T17:59:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.964+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881f7ab0 con 0x7f36881f7630 2024-03-28T17:59:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.965+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368804f400 con 0x7f36881f7630 2024-03-28T17:59:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.966+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f367000f450 con 0x7f36881f7630 2024-03-28T17:59:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.967+0000 7f367ea00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 0x7f366406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.967+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f367008b2c0 con 0x7f36881f7630 2024-03-28T17:59:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.967+0000 7f3686a00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 0x7f366406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:33.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.969+0000 7f3686a00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 0x7f366406e0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3678033000 tx=0x7f3678009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:33.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:33.971+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3670019030 con 0x7f36881f7630 2024-03-28T17:59:34.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.180+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36880b2330 con 0x7f36881f7630 2024-03-28T17:59:34.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.181+0000 7f367ea00640 1 -- 172.21.15.62:0/1277410028 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36700596c0 con 0x7f36881f7630 2024-03-28T17:59:34.181 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 msgr2=0x7f366406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 0x7f366406e0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3678033000 tx=0x7f3678009030 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 msgr2=0x7f36880b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3670011630 tx=0x7f3670011af0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 shutdown_connections 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f366406bc00 0x7f366406e0c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36880b1f50 0x7f36881f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36881f7630 0x7f36880b1a10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 --2- 172.21.15.62:0/1277410028 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f366c0039e0 0x7f36880c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 >> 172.21.15.62:0/1277410028 conn(0x7f3688066dd0 msgr2=0x7f36880b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 shutdown_connections 2024-03-28T17:59:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.184+0000 7f3687e00640 1 -- 172.21.15.62:0/1277410028 wait complete. 2024-03-28T17:59:34.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.414+0000 7f8e61600640 1 Processor -- start 2024-03-28T17:59:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.414+0000 7f8e61600640 1 -- start start 2024-03-28T17:59:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 0x7f8e5c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c15fa00 0x7f8e5c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8e5c15fe80 0x7f8e5c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c1603c0 con 0x7f8e5c15fa00 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c1606a0 con 0x7f8e5c15fe80 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e61600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b3cb0 con 0x7f8e5c152f90 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e5a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c15fa00 0x7f8e5c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.416+0000 7f8e5a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c15fa00 0x7f8e5c0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46198/0 (socket says 172.21.15.62:46198) 2024-03-28T17:59:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 -- 172.21.15.62:0/4252198035 learned_addr learned my addr 172.21.15.62:0/4252198035 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5b400640 1 --2- 172.21.15.62:0/4252198035 >> v2:172.21.15.81:3300/0 conn(0x7f8e5c15fe80 0x7f8e5c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 0x7f8e5c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 -- 172.21.15.62:0/4252198035 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 msgr2=0x7f8e5c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 0x7f8e5c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 -- 172.21.15.62:0/4252198035 >> v2:172.21.15.81:3300/0 conn(0x7f8e5c15fe80 msgr2=0x7f8e5c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 --2- 172.21.15.62:0/4252198035 >> v2:172.21.15.81:3300/0 conn(0x7f8e5c15fe80 0x7f8e5c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5a000640 1 -- 172.21.15.62:0/4252198035 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0b3f90 con 0x7f8e5c15fa00 2024-03-28T17:59:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.417+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 0x7f8e5c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.418+0000 7f8e5a000640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c0cee80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c073090 tx=0x7f8e4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe7255720ed0495 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.419+0000 7f8e59600640 1 -- 172.21.15.62:0/4252198035 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48035020 con 0x7f8e5c15fa00 2024-03-28T17:59:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.419+0000 7f8e59600640 1 -- 172.21.15.62:0/4252198035 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48031870 con 0x7f8e5c15fa00 2024-03-28T17:59:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.419+0000 7f8e59600640 1 -- 172.21.15.62:0/4252198035 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48031c40 con 0x7f8e5c15fa00 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.419+0000 7f8e61600640 1 -- 172.21.15.62:0/4252198035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 msgr2=0x7f8e5c0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.419+0000 7f8e61600640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c0cee80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c073090 tx=0x7f8e4802b910 comp rx=0 tx=0).stop 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 -- 172.21.15.62:0/4252198035 shutdown_connections 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 --2- 172.21.15.62:0/4252198035 >> v2:172.21.15.81:3300/0 conn(0x7f8e5c15fe80 0x7f8e5c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c0cee80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 --2- 172.21.15.62:0/4252198035 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c152f90 0x7f8e5c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 -- 172.21.15.62:0/4252198035 >> 172.21.15.62:0/4252198035 conn(0x7f8e5c066fb0 msgr2=0x7f8e5c0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 -- 172.21.15.62:0/4252198035 shutdown_connections 2024-03-28T17:59:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.420+0000 7f8e61600640 1 -- 172.21.15.62:0/4252198035 wait complete. 2024-03-28T17:59:34.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.422+0000 7f8e61600640 1 Processor -- start 2024-03-28T17:59:34.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.422+0000 7f8e61600640 1 -- start start 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e5c152f90 0x7f8e5c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c15fe80 0x7f8e5c14e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b4630 con 0x7f8e5c15fa00 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b44b0 con 0x7f8e5c152f90 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.423+0000 7f8e61600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b47b0 con 0x7f8e5c15fe80 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46210/0 (socket says 172.21.15.62:46210) 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 -- 172.21.15.62:0/2638011000 learned_addr learned my addr 172.21.15.62:0/2638011000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e5c152f90 0x7f8e5c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5b400640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c15fe80 0x7f8e5c14e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 -- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c15fe80 msgr2=0x7f8e5c14e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c15fe80 0x7f8e5c14e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 -- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e5c152f90 msgr2=0x7f8e5c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e5c152f90 0x7f8e5c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.424+0000 7f8e5a000640 1 -- 172.21.15.62:0/2638011000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c09cb30 con 0x7f8e5c15fa00 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e5a000640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e48000c00 tx=0x7f8e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48035040 con 0x7f8e5c15fa00 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48030d00 con 0x7f8e5c15fa00 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e480456a0 con 0x7f8e5c15fa00 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c09ce10 con 0x7f8e5c15fa00 2024-03-28T17:59:34.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.425+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c09d3d0 con 0x7f8e5c15fa00 2024-03-28T17:59:34.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.427+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0680c0 con 0x7f8e5c15fa00 2024-03-28T17:59:34.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.428+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8e48038020 con 0x7f8e5c15fa00 2024-03-28T17:59:34.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.429+0000 7f8e52a00640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 0x7f8e3c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.429+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e480acd90 con 0x7f8e5c15fa00 2024-03-28T17:59:34.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.430+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 0x7f8e3c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.431+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 0x7f8e3c06e0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c006b10 tx=0x7f8e4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.434+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8e4807b870 con 0x7f8e5c15fa00 2024-03-28T17:59:34.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.637+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e5c0b3500 con 0x7f8e5c15fa00 2024-03-28T17:59:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.639+0000 7f8e52a00640 1 -- 172.21.15.62:0/2638011000 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f8e4807b170 con 0x7f8e5c15fa00 2024-03-28T17:59:34.640 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:34.640 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:58:54.988474+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:34.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 msgr2=0x7f8e3c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 0x7f8e3c06e0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c006b10 tx=0x7f8e4c008040 comp rx=0 tx=0).stop 2024-03-28T17:59:34.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 msgr2=0x7f8e5c14e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e48000c00 tx=0x7f8e480046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 shutdown_connections 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f8e3c06bc00 0x7f8e3c06e0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8e5c15fe80 0x7f8e5c14e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c15fa00 0x7f8e5c14e0e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 --2- 172.21.15.62:0/2638011000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e5c152f90 0x7f8e5c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 >> 172.21.15.62:0/2638011000 conn(0x7f8e5c066fb0 msgr2=0x7f8e5c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 shutdown_connections 2024-03-28T17:59:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.642+0000 7f8e61600640 1 -- 172.21.15.62:0/2638011000 wait complete. 2024-03-28T17:59:34.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:34.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:34.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.882+0000 7ff5a3e00640 1 Processor -- start 2024-03-28T17:59:34.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.882+0000 7ff5a3e00640 1 -- start start 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff5a4160190 0x7ff5a415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff5a40b44f0 0x7ff5a40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40b4ea0 con 0x7ff5a4160190 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a415cfd0 con 0x7ff5a40b44f0 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff5a3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a415bec0 con 0x7ff5a40aee00 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff59be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff5a4160190 0x7ff5a415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff59be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff5a4160190 0x7ff5a415ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46222/0 (socket says 172.21.15.62:46222) 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.884+0000 7ff59be00640 1 -- 172.21.15.62:0/3852712273 learned_addr learned my addr 172.21.15.62:0/3852712273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff5a2a00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff5a3400640 1 --2- 172.21.15.62:0/3852712273 >> v2:172.21.15.81:3300/0 conn(0x7ff5a40b44f0 0x7ff5a40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff59be00640 1 -- 172.21.15.62:0/3852712273 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 msgr2=0x7ff5a40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff59be00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a40b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff59be00640 1 -- 172.21.15.62:0/3852712273 >> v2:172.21.15.81:3300/0 conn(0x7ff5a40b44f0 msgr2=0x7ff5a40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff59be00640 1 --2- 172.21.15.62:0/3852712273 >> v2:172.21.15.81:3300/0 conn(0x7ff5a40b44f0 0x7ff5a40b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.885+0000 7ff59be00640 1 -- 172.21.15.62:0/3852712273 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40b1bf0 con 0x7ff5a4160190 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.886+0000 7ff59be00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a415ca90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4067d90 tx=0x7ff58c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d204ecd53aa3f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.886+0000 7ff5a2000640 1 -- 172.21.15.62:0/3852712273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c035020 con 0x7ff5a4160190 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.886+0000 7ff5a2000640 1 -- 172.21.15.62:0/3852712273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c02d040 con 0x7ff5a4160190 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.886+0000 7ff5a2000640 1 -- 172.21.15.62:0/3852712273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c031a80 con 0x7ff5a4160190 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 -- 172.21.15.62:0/3852712273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 msgr2=0x7ff5a415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a415ca90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4067d90 tx=0x7ff58c02b900 comp rx=0 tx=0).stop 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 -- 172.21.15.62:0/3852712273 shutdown_connections 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/3852712273 >> v2:172.21.15.81:3300/0 conn(0x7ff5a40b44f0 0x7ff5a40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a415ca90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/3852712273 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a40b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 -- 172.21.15.62:0/3852712273 >> 172.21.15.62:0/3852712273 conn(0x7ff5a4066dd0 msgr2=0x7ff5a40671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 -- 172.21.15.62:0/3852712273 shutdown_connections 2024-03-28T17:59:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.887+0000 7ff5a3e00640 1 -- 172.21.15.62:0/3852712273 wait complete. 2024-03-28T17:59:34.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.889+0000 7ff5a3e00640 1 Processor -- start 2024-03-28T17:59:34.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.890+0000 7ff5a3e00640 1 -- start start 2024-03-28T17:59:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5a40b44f0 0x7ff5a41e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a415f990 con 0x7ff5a4160190 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a415f6b0 con 0x7ff5a40b44f0 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a415fc70 con 0x7ff5a40aee00 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46236/0 (socket says 172.21.15.62:46236) 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a3400640 1 -- 172.21.15.62:0/2358168653 learned_addr learned my addr 172.21.15.62:0/2358168653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.891+0000 7ff5a2a00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff59be00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5a40b44f0 0x7ff5a41e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 -- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 msgr2=0x7ff5a414c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a414c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 -- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5a40b44f0 msgr2=0x7ff5a41e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5a40b44f0 0x7ff5a41e6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 -- 172.21.15.62:0/2358168653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a41e6ac0 con 0x7ff5a4160190 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3400640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00cd70 tx=0x7ff59c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c003b90 con 0x7ff5a4160190 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c003d30 con 0x7ff5a4160190 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c004240 con 0x7ff5a4160190 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a41e6e00 con 0x7ff5a4160190 2024-03-28T17:59:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.892+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41e7640 con 0x7ff5a4160190 2024-03-28T17:59:34.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.894+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a404f400 con 0x7ff5a4160190 2024-03-28T17:59:34.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.894+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff59c0043e0 con 0x7ff5a4160190 2024-03-28T17:59:34.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.895+0000 7ff59b400640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 0x7ff58006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:34.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.895+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff59c08b530 con 0x7ff5a4160190 2024-03-28T17:59:34.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.896+0000 7ff5a2a00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 0x7ff58006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.897+0000 7ff5a2a00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 0x7ff58006e0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff594006fd0 tx=0x7ff594008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:34.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:34.900+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff5a41e6e00 con 0x7ff5a4160190 2024-03-28T17:59:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.103+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5a40b33d0 con 0x7ff5a4160190 2024-03-28T17:59:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.104+0000 7ff59b400640 1 -- 172.21.15.62:0/2358168653 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff59c05a000 con 0x7ff5a4160190 2024-03-28T17:59:35.104 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 msgr2=0x7ff58006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 0x7ff58006e0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff594006fd0 tx=0x7ff594008040 comp rx=0 tx=0).stop 2024-03-28T17:59:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 msgr2=0x7ff5a40ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00cd70 tx=0x7ff59c00a660 comp rx=0 tx=0).stop 2024-03-28T17:59:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 shutdown_connections 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff58006bc20 0x7ff58006e0e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4160190 0x7ff5a40ce380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5a40b44f0 0x7ff5a41e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 --2- 172.21.15.62:0/2358168653 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff5a40aee00 0x7ff5a414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 >> 172.21.15.62:0/2358168653 conn(0x7ff5a4066dd0 msgr2=0x7ff5a40b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 shutdown_connections 2024-03-28T17:59:35.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.107+0000 7ff5a3e00640 1 -- 172.21.15.62:0/2358168653 wait complete. 2024-03-28T17:59:35.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.351+0000 7f77cca00640 1 Processor -- start 2024-03-28T17:59:35.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.351+0000 7f77cca00640 1 -- start start 2024-03-28T17:59:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.352+0000 7f77cca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f77c806e210 0x7f77c815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77cca00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f77c815eae0 0x7f77c80713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77cca00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c815be10 con 0x7f77c8158550 2024-03-28T17:59:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77cca00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c8158930 con 0x7f77c806e210 2024-03-28T17:59:35.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77cca00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c815b050 con 0x7f77c815eae0 2024-03-28T17:59:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77c6800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f77c8158550 0x7f77c806db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41850/0 (socket says 172.21.15.62:41850) 2024-03-28T17:59:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77bde00640 1 --2- 172.21.15.62:0/309783420 >> v2:172.21.15.62:3300/0 conn(0x7f77c806e210 0x7f77c815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77c6800640 1 -- 172.21.15.62:0/309783420 learned_addr learned my addr 172.21.15.62:0/309783420 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:35.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77c7200640 1 --2- 172.21.15.62:0/309783420 >> v2:172.21.15.81:3300/0 conn(0x7f77c815eae0 0x7f77c80713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77bde00640 1 -- 172.21.15.62:0/309783420 >> v2:172.21.15.81:3300/0 conn(0x7f77c815eae0 msgr2=0x7f77c80713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77bde00640 1 --2- 172.21.15.62:0/309783420 >> v2:172.21.15.81:3300/0 conn(0x7f77c815eae0 0x7f77c80713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.353+0000 7f77bde00640 1 -- 172.21.15.62:0/309783420 >> v1:172.21.15.62:6789/0 conn(0x7f77c8158550 legacy=0x7f77c806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.354+0000 7f77bde00640 1 -- 172.21.15.62:0/309783420 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c815b330 con 0x7f77c806e210 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.354+0000 7f77c5e00640 1 -- 172.21.15.62:0/309783420 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267396167 0 0) 0x7f77c815be10 con 0x7f77c8158550 2024-03-28T17:59:35.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.355+0000 7f77bde00640 1 --2- 172.21.15.62:0/309783420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c806e210 0x7f77c815e5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f77c815af50 tx=0x7f77b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f7a8ac12ea86d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.355+0000 7f77c5e00640 1 -- 172.21.15.62:0/309783420 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0035020 con 0x7f77c806e210 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.355+0000 7f77c5e00640 1 -- 172.21.15.62:0/309783420 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b0031830 con 0x7f77c806e210 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.355+0000 7f77c5e00640 1 -- 172.21.15.62:0/309783420 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0031c00 con 0x7f77c806e210 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 -- 172.21.15.62:0/309783420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c806e210 msgr2=0x7f77c815e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 --2- 172.21.15.62:0/309783420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c806e210 0x7f77c815e5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f77c815af50 tx=0x7f77b002b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 -- 172.21.15.62:0/309783420 shutdown_connections 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 --2- 172.21.15.62:0/309783420 >> v2:172.21.15.81:3300/0 conn(0x7f77c815eae0 0x7f77c80713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 --2- 172.21.15.62:0/309783420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c806e210 0x7f77c815e5a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 -- 172.21.15.62:0/309783420 >> 172.21.15.62:0/309783420 conn(0x7f77c8066dd0 msgr2=0x7f77c80671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 -- 172.21.15.62:0/309783420 shutdown_connections 2024-03-28T17:59:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.356+0000 7f77cca00640 1 -- 172.21.15.62:0/309783420 wait complete. 2024-03-28T17:59:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.358+0000 7f77cca00640 1 Processor -- start 2024-03-28T17:59:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.358+0000 7f77cca00640 1 -- start start 2024-03-28T17:59:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77c806e210 0x7f77c80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 0x7f77c80ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c815f310 con 0x7f77c8158550 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c815b8f0 con 0x7f77c815eae0 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77cca00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c815ef00 con 0x7f77c806e210 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77bde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77bde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46258/0 (socket says 172.21.15.62:46258) 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77bde00640 1 -- 172.21.15.62:0/2365995898 learned_addr learned my addr 172.21.15.62:0/2365995898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.359+0000 7f77c6800640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77c806e210 0x7f77c80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77c7200640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 0x7f77c80ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 -- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77c806e210 msgr2=0x7f77c80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77c806e210 0x7f77c80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 -- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 msgr2=0x7f77c80ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 0x7f77c80ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 -- 172.21.15.62:0/2365995898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80ada70 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77c7200640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 0x7f77c80ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.360+0000 7f77bde00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009890 tx=0x7f77b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.361+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0035070 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.361+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b0004250 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.361+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b003a630 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.361+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c80ce300 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.361+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c80add10 con 0x7f77c8158550 2024-03-28T17:59:35.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.362+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c80adf10 con 0x7f77c8158550 2024-03-28T17:59:35.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.363+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f77b0038020 con 0x7f77c8158550 2024-03-28T17:59:35.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.364+0000 7f77bf400640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 0x7f77a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.364+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f77b00ac6b0 con 0x7f77c8158550 2024-03-28T17:59:35.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.365+0000 7f77c6800640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 0x7f77a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.366+0000 7f77c6800640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 0x7f77a406e090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f77b80012e0 tx=0x7f77b8001700 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.369+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f77c80adf10 con 0x7f77c8158550 2024-03-28T17:59:35.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.572+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77c804f400 con 0x7f77c8158550 2024-03-28T17:59:35.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.573+0000 7f77bf400640 1 -- 172.21.15.62:0/2365995898 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77b0040070 con 0x7f77c8158550 2024-03-28T17:59:35.573 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 msgr2=0x7f77a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 0x7f77a406e090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f77b80012e0 tx=0x7f77b8001700 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 msgr2=0x7f77c8145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009890 tx=0x7f77b00046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 shutdown_connections 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f77a406bbd0 0x7f77a406e090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77c815eae0 0x7f77c80ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c8158550 0x7f77c8145970 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 --2- 172.21.15.62:0/2365995898 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f77c806e210 0x7f77c80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.575+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 >> 172.21.15.62:0/2365995898 conn(0x7f77c8066dd0 msgr2=0x7f77c80673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:35.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.576+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 shutdown_connections 2024-03-28T17:59:35.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.576+0000 7f77cca00640 1 -- 172.21.15.62:0/2365995898 wait complete. 2024-03-28T17:59:35.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:35.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.808+0000 7f2dbbc00640 1 Processor -- start 2024-03-28T17:59:35.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.808+0000 7f2dbbc00640 1 -- start start 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.809+0000 7f2dbbc00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2db40ae720 0x7f2db40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.809+0000 7f2dbbc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.809+0000 7f2dbbc00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db415d1b0 con 0x7f2db4072690 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.809+0000 7f2dbbc00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db40af040 con 0x7f2db40ae720 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.809+0000 7f2dbbc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4070d50 con 0x7f2db4072b00 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.810+0000 7f2db9200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2db40ae720 0x7f2db40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.810+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.810+0000 7f2db9200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2db40ae720 0x7f2db40aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34116/0 (socket says 172.21.15.62:34116) 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.810+0000 7f2db9200640 1 -- 172.21.15.62:0/2772630360 learned_addr learned my addr 172.21.15.62:0/2772630360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.811+0000 7f2db9200640 1 -- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 msgr2=0x7f2db415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.811+0000 7f2db9200640 1 --2- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.811+0000 7f2db9200640 1 -- 172.21.15.62:0/2772630360 >> v1:172.21.15.81:6789/0 conn(0x7f2db4072690 legacy=0x7f2db415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.811+0000 7f2db9200640 1 -- 172.21.15.62:0/2772630360 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4071030 con 0x7f2db40ae720 2024-03-28T17:59:35.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.812+0000 7f2db9200640 1 --2- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db40ae720 0x7f2db40aeb00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5ef3e73446c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2db3400640 1 -- 172.21.15.62:0/2772630360 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8034070 con 0x7f2db40ae720 2024-03-28T17:59:35.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2db3400640 1 -- 172.21.15.62:0/2772630360 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da8030840 con 0x7f2db40ae720 2024-03-28T17:59:35.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2db3400640 1 -- 172.21.15.62:0/2772630360 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8030c10 con 0x7f2db40ae720 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 -- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db40ae720 msgr2=0x7f2db40aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db40ae720 0x7f2db40aeb00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da802b980 comp rx=0 tx=0).stop 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 -- 172.21.15.62:0/2772630360 shutdown_connections 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/2772630360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db40ae720 0x7f2db40aeb00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 -- 172.21.15.62:0/2772630360 >> 172.21.15.62:0/2772630360 conn(0x7f2db40b1650 msgr2=0x7f2db40b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 -- 172.21.15.62:0/2772630360 shutdown_connections 2024-03-28T17:59:35.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.813+0000 7f2dbbc00640 1 -- 172.21.15.62:0/2772630360 wait complete. 2024-03-28T17:59:35.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.815+0000 7f2dbbc00640 1 Processor -- start 2024-03-28T17:59:35.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.815+0000 7f2dbbc00640 1 -- start start 2024-03-28T17:59:35.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 0x7f2db414c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db414f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40718d0 con 0x7f2db40ae720 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40715f0 con 0x7f2db4072690 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.816+0000 7f2dbbc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4070a40 con 0x7f2db4072b00 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46260/0 (socket says 172.21.15.62:46260) 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 -- 172.21.15.62:0/394863026 learned_addr learned my addr 172.21.15.62:0/394863026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9200640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 0x7f2db414c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db3e00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db414f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 -- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 msgr2=0x7f2db414f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db414f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 -- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 msgr2=0x7f2db414c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:35.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 0x7f2db414c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:35.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9c00640 1 -- 172.21.15.62:0/394863026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db414d080 con 0x7f2db40ae720 2024-03-28T17:59:35.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.817+0000 7f2db9200640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 0x7f2db414c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:35.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2db9c00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00ef90 tx=0x7f2dac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac018020 con 0x7f2db40ae720 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac014480 con 0x7f2db40ae720 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac014870 con 0x7f2db40ae720 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db414d360 con 0x7f2db40ae720 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.818+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db414d920 con 0x7f2db40ae720 2024-03-28T17:59:35.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.819+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db40680c0 con 0x7f2db40ae720 2024-03-28T17:59:35.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.821+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2dac005d10 con 0x7f2db40ae720 2024-03-28T17:59:35.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.821+0000 7f2db0c00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 0x7f2d9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:35.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.822+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2dac08ba80 con 0x7f2db40ae720 2024-03-28T17:59:35.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.822+0000 7f2db9200640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 0x7f2d9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:35.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.823+0000 7f2db9200640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 0x7f2d9406e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2da8036000 tx=0x7f2da803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:35.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:35.827+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2dac08de20 con 0x7f2db40ae720 2024-03-28T17:59:36.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.029+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2db40b2420 con 0x7f2db40ae720 2024-03-28T17:59:36.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.030+0000 7f2db0c00640 1 -- 172.21.15.62:0/394863026 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2dac022030 con 0x7f2db40ae720 2024-03-28T17:59:36.030 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.032+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 msgr2=0x7f2d9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.032+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 0x7f2d9406e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2da8036000 tx=0x7f2da803b040 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.032+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 msgr2=0x7f2db406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.032+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00ef90 tx=0x7f2dac00c490 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.032+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 shutdown_connections 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2d9406bbd0 0x7f2d9406e090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db40ae720 0x7f2db406d890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2db4072b00 0x7f2db414f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 --2- 172.21.15.62:0/394863026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4072690 0x7f2db414c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 >> 172.21.15.62:0/394863026 conn(0x7f2db40b1650 msgr2=0x7f2db40af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 shutdown_connections 2024-03-28T17:59:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.033+0000 7f2dbbc00640 1 -- 172.21.15.62:0/394863026 wait complete. 2024-03-28T17:59:36.103 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-28T17:59:36.104 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:36.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.337+0000 7fbb6f200640 1 Processor -- start 2024-03-28T17:59:36.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.338+0000 7fbb6f200640 1 -- start start 2024-03-28T17:59:36.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.339+0000 7fbb6f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb701456d0 0x7fbb7015f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.339+0000 7fbb6f200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb70072ab0 0x7fbb7015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.339+0000 7fbb6f200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb700cebb0 con 0x7fbb7006cd80 2024-03-28T17:59:36.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.339+0000 7fbb6f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700cee90 con 0x7fbb701456d0 2024-03-28T17:59:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.339+0000 7fbb6f200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700cf170 con 0x7fbb70072ab0 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb701456d0 0x7fbb7015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb701456d0 0x7fbb7015f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46272/0 (socket says 172.21.15.62:46272) 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 -- 172.21.15.62:0/1205433674 learned_addr learned my addr 172.21.15.62:0/1205433674 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6d400640 1 --2- 172.21.15.62:0/1205433674 >> v2:172.21.15.81:3300/0 conn(0x7fbb70072ab0 0x7fbb7015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 -- 172.21.15.62:0/1205433674 >> v2:172.21.15.81:3300/0 conn(0x7fbb70072ab0 msgr2=0x7fbb7015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 --2- 172.21.15.62:0/1205433674 >> v2:172.21.15.81:3300/0 conn(0x7fbb70072ab0 0x7fbb7015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 -- 172.21.15.62:0/1205433674 >> v1:172.21.15.81:6789/0 conn(0x7fbb7006cd80 legacy=0x7fbb7006d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.340+0000 7fbb6de00640 1 -- 172.21.15.62:0/1205433674 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700cf450 con 0x7fbb701456d0 2024-03-28T17:59:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.342+0000 7fbb6de00640 1 --2- 172.21.15.62:0/1205433674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb7015f980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb700724a0 tx=0x7fbb6002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87463ae28a714346 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.342+0000 7fbb6ca00640 1 -- 172.21.15.62:0/1205433674 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6002e070 con 0x7fbb701456d0 2024-03-28T17:59:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.342+0000 7fbb6ca00640 1 -- 172.21.15.62:0/1205433674 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60037070 con 0x7fbb701456d0 2024-03-28T17:59:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.342+0000 7fbb6ca00640 1 -- 172.21.15.62:0/1205433674 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60031a00 con 0x7fbb701456d0 2024-03-28T17:59:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 -- 172.21.15.62:0/1205433674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 msgr2=0x7fbb7015f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 --2- 172.21.15.62:0/1205433674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb7015f980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb700724a0 tx=0x7fbb6002b880 comp rx=0 tx=0).stop 2024-03-28T17:59:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 -- 172.21.15.62:0/1205433674 shutdown_connections 2024-03-28T17:59:36.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 --2- 172.21.15.62:0/1205433674 >> v2:172.21.15.81:3300/0 conn(0x7fbb70072ab0 0x7fbb7015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 --2- 172.21.15.62:0/1205433674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb7015f980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 -- 172.21.15.62:0/1205433674 >> 172.21.15.62:0/1205433674 conn(0x7fbb700b1730 msgr2=0x7fbb700b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:36.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 -- 172.21.15.62:0/1205433674 shutdown_connections 2024-03-28T17:59:36.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.343+0000 7fbb6f200640 1 -- 172.21.15.62:0/1205433674 wait complete. 2024-03-28T17:59:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.345+0000 7fbb6f200640 1 Processor -- start 2024-03-28T17:59:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.345+0000 7fbb6f200640 1 -- start start 2024-03-28T17:59:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7006cd80 0x7fbb7013ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb70072ab0 0x7fbb70144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70071a50 con 0x7fbb701456d0 2024-03-28T17:59:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700cf730 con 0x7fbb70072ab0 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70071bd0 con 0x7fbb7006cd80 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46280/0 (socket says 172.21.15.62:46280) 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6e800640 1 -- 172.21.15.62:0/3264479630 learned_addr learned my addr 172.21.15.62:0/3264479630 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6d400640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb70072ab0 0x7fbb70144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.347+0000 7fbb6de00640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7006cd80 0x7fbb7013ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 -- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7006cd80 msgr2=0x7fbb7013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7006cd80 0x7fbb7013ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 -- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb70072ab0 msgr2=0x7fbb70144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb70072ab0 0x7fbb70144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 -- 172.21.15.62:0/3264479630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700ab6d0 con 0x7fbb701456d0 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb6e800640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c780 tx=0x7fbb6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68004b60 con 0x7fbb701456d0 2024-03-28T17:59:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68004d00 con 0x7fbb701456d0 2024-03-28T17:59:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.348+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800aa30 con 0x7fbb701456d0 2024-03-28T17:59:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.349+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb70155d70 con 0x7fbb701456d0 2024-03-28T17:59:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.349+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb70156330 con 0x7fbb701456d0 2024-03-28T17:59:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.350+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70073490 con 0x7fbb701456d0 2024-03-28T17:59:36.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.351+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbb6800abd0 con 0x7fbb701456d0 2024-03-28T17:59:36.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.352+0000 7fbb66000640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 0x7fbb5006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.352+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbb6808ac90 con 0x7fbb701456d0 2024-03-28T17:59:36.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.353+0000 7fbb6de00640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 0x7fbb5006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.354+0000 7fbb6de00640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 0x7fbb5006e0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb60036000 tx=0x7fbb6003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.358+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbb68020070 con 0x7fbb701456d0 2024-03-28T17:59:36.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.560+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb7006e6e0 con 0x7fbb701456d0 2024-03-28T17:59:36.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.563+0000 7fbb66000640 1 -- 172.21.15.62:0/3264479630 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7fbb68059010 con 0x7fbb701456d0 2024-03-28T17:59:36.563 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:36.564 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 msgr2=0x7fbb5006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 0x7fbb5006e0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb60036000 tx=0x7fbb6003d040 comp rx=0 tx=0).stop 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 msgr2=0x7fbb701571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c780 tx=0x7fbb6800cc40 comp rx=0 tx=0).stop 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 shutdown_connections 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbb5006bc00 0x7fbb5006e0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb701456d0 0x7fbb701571b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb70072ab0 0x7fbb70144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 --2- 172.21.15.62:0/3264479630 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbb7006cd80 0x7fbb7013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 >> 172.21.15.62:0/3264479630 conn(0x7fbb700b1730 msgr2=0x7fbb700cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:36.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 shutdown_connections 2024-03-28T17:59:36.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.566+0000 7fbb6f200640 1 -- 172.21.15.62:0/3264479630 wait complete. 2024-03-28T17:59:36.587 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:36.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.805+0000 7fb86de00640 1 Processor -- start 2024-03-28T17:59:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.805+0000 7fb86de00640 1 -- start start 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb86de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb86806f580 0x7fb8680cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb86de00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb86814bc90 0x7fb8680c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb86de00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86806b460 con 0x7fb86814c0f0 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb86de00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86806c070 con 0x7fb86806f580 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb86de00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86806b740 con 0x7fb86814bc90 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb866a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb86806f580 0x7fb8680cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb867400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb86814c0f0 0x7fb86806c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41862/0 (socket says 172.21.15.62:41862) 2024-03-28T17:59:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb867400640 1 -- 172.21.15.62:0/1923211640 learned_addr learned my addr 172.21.15.62:0/1923211640 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.807+0000 7fb866000640 1 --2- 172.21.15.62:0/1923211640 >> v2:172.21.15.81:3300/0 conn(0x7fb86814bc90 0x7fb8680c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.808+0000 7fb866a00640 1 -- 172.21.15.62:0/1923211640 >> v2:172.21.15.81:3300/0 conn(0x7fb86814bc90 msgr2=0x7fb8680c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.808+0000 7fb866a00640 1 --2- 172.21.15.62:0/1923211640 >> v2:172.21.15.81:3300/0 conn(0x7fb86814bc90 0x7fb8680c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.808+0000 7fb866a00640 1 -- 172.21.15.62:0/1923211640 >> v1:172.21.15.62:6789/0 conn(0x7fb86814c0f0 legacy=0x7fb86806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.808+0000 7fb866a00640 1 -- 172.21.15.62:0/1923211640 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86806ba20 con 0x7fb86806f580 2024-03-28T17:59:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.808+0000 7fb865600640 1 -- 172.21.15.62:0/1923211640 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616663326 0 0) 0x7fb86806b460 con 0x7fb86814c0f0 2024-03-28T17:59:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.809+0000 7fb866a00640 1 --2- 172.21.15.62:0/1923211640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86806f580 0x7fb8680cf780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8681660c0 tx=0x7fb85802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf673d1e685b6ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.809+0000 7fb865600640 1 -- 172.21.15.62:0/1923211640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858039070 con 0x7fb86806f580 2024-03-28T17:59:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.809+0000 7fb865600640 1 -- 172.21.15.62:0/1923211640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85802cc70 con 0x7fb86806f580 2024-03-28T17:59:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.809+0000 7fb865600640 1 -- 172.21.15.62:0/1923211640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858034660 con 0x7fb86806f580 2024-03-28T17:59:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 -- 172.21.15.62:0/1923211640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86806f580 msgr2=0x7fb8680cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 --2- 172.21.15.62:0/1923211640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86806f580 0x7fb8680cf780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8681660c0 tx=0x7fb85802bec0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 -- 172.21.15.62:0/1923211640 shutdown_connections 2024-03-28T17:59:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 --2- 172.21.15.62:0/1923211640 >> v2:172.21.15.81:3300/0 conn(0x7fb86814bc90 0x7fb8680c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 --2- 172.21.15.62:0/1923211640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86806f580 0x7fb8680cf780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 -- 172.21.15.62:0/1923211640 >> 172.21.15.62:0/1923211640 conn(0x7fb8680b3700 msgr2=0x7fb8680b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 -- 172.21.15.62:0/1923211640 shutdown_connections 2024-03-28T17:59:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.810+0000 7fb86de00640 1 -- 172.21.15.62:0/1923211640 wait complete. 2024-03-28T17:59:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.812+0000 7fb86de00640 1 Processor -- start 2024-03-28T17:59:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.812+0000 7fb86de00640 1 -- start start 2024-03-28T17:59:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 0x7fb8681f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb86814bc90 0x7fb8680b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86806b0a0 con 0x7fb86814c0f0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680cfcc0 con 0x7fb86814bc90 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb86de00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86806b220 con 0x7fb86806f580 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb867400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb867400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46300/0 (socket says 172.21.15.62:46300) 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb867400640 1 -- 172.21.15.62:0/720233882 learned_addr learned my addr 172.21.15.62:0/720233882 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb866000640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb86814bc90 0x7fb8680b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.813+0000 7fb866a00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 0x7fb8681f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 -- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 msgr2=0x7fb8681f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 0x7fb8681f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 -- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb86814bc90 msgr2=0x7fb8680b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb86814bc90 0x7fb8680b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 -- 172.21.15.62:0/720233882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680b52f0 con 0x7fb86814c0f0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb866a00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 0x7fb8681f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb867400640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c480 tx=0x7fb86000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860018070 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860007500 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8680b55d0 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860004a20 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.814+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8680b5b90 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.815+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8680687d0 con 0x7fb86814c0f0 2024-03-28T17:59:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.816+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb86001b070 con 0x7fb86814c0f0 2024-03-28T17:59:36.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.817+0000 7fb85f400640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 0x7fb84406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:36.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.817+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb860089e10 con 0x7fb86814c0f0 2024-03-28T17:59:36.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.818+0000 7fb866a00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 0x7fb84406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:36.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.819+0000 7fb866a00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 0x7fb84406e090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb858036000 tx=0x7fb858030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:36.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:36.822+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb86008d030 con 0x7fb86814c0f0 2024-03-28T17:59:37.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.028+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8680b7b30 con 0x7fb86814c0f0 2024-03-28T17:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.029+0000 7fb85f400640 1 -- 172.21.15.62:0/720233882 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb860058870 con 0x7fb86814c0f0 2024-03-28T17:59:37.029 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.032+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 msgr2=0x7fb84406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.032+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 0x7fb84406e090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb858036000 tx=0x7fb858030000 comp rx=0 tx=0).stop 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.032+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 msgr2=0x7fb8680b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.032+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c480 tx=0x7fb86000c940 comp rx=0 tx=0).stop 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 shutdown_connections 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb84406bbd0 0x7fb84406e090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86814c0f0 0x7fb8680b23e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb86814bc90 0x7fb8680b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 --2- 172.21.15.62:0/720233882 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb86806f580 0x7fb8681f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 >> 172.21.15.62:0/720233882 conn(0x7fb8680b3700 msgr2=0x7fb868161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 shutdown_connections 2024-03-28T17:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.033+0000 7fb86de00640 1 -- 172.21.15.62:0/720233882 wait complete. 2024-03-28T17:59:37.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:37.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.269+0000 7f946d400640 1 Processor -- start 2024-03-28T17:59:37.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.270+0000 7f946d400640 1 -- start start 2024-03-28T17:59:37.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.271+0000 7f946d400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.271+0000 7f946d400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.271+0000 7f946d400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94680cf570 con 0x7f94680b3210 2024-03-28T17:59:37.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.271+0000 7f946d400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9468161bb0 con 0x7f946815e2c0 2024-03-28T17:59:37.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.271+0000 7f946d400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9468161e90 con 0x7f94680cec50 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:34120/0 (socket says 172.21.15.62:34120) 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 -- 172.21.15.62:0/773615256 learned_addr learned my addr 172.21.15.62:0/773615256 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34162/0 (socket says 172.21.15.62:34162) 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 -- 172.21.15.62:0/773615256 >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 msgr2=0x7f94680cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 --2- 172.21.15.62:0/773615256 >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 -- 172.21.15.62:0/773615256 >> v1:172.21.15.81:6789/0 conn(0x7f94680b3210 legacy=0x7f94680b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467e00640 1 -- 172.21.15.62:0/773615256 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468162170 con 0x7f94680cec50 2024-03-28T17:59:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.272+0000 7f9467400640 1 --2- 172.21.15.62:0/773615256 >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f9467e00640 1 --2- 172.21.15.62:0/773615256 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9458009e50 tx=0x7f945802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b34d74747a15ae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f9466a00640 1 -- 172.21.15.62:0/773615256 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945802cb00 con 0x7f94680cec50 2024-03-28T17:59:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f9466a00640 1 -- 172.21.15.62:0/773615256 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945802cca0 con 0x7f94680cec50 2024-03-28T17:59:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f9466a00640 1 -- 172.21.15.62:0/773615256 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458034610 con 0x7f94680cec50 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f946d400640 1 -- 172.21.15.62:0/773615256 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 msgr2=0x7f94680ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.274+0000 7f946d400640 1 --2- 172.21.15.62:0/773615256 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9458009e50 tx=0x7f945802bf60 comp rx=0 tx=0).stop 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 -- 172.21.15.62:0/773615256 shutdown_connections 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 --2- 172.21.15.62:0/773615256 >> v2:172.21.15.81:3300/0 conn(0x7f946815e2c0 0x7f94680cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 --2- 172.21.15.62:0/773615256 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94680cec50 0x7f94680ce570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 -- 172.21.15.62:0/773615256 >> 172.21.15.62:0/773615256 conn(0x7f9468066fb0 msgr2=0x7f94680673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 -- 172.21.15.62:0/773615256 shutdown_connections 2024-03-28T17:59:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.275+0000 7f946d400640 1 -- 172.21.15.62:0/773615256 wait complete. 2024-03-28T17:59:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.276+0000 7f946d400640 1 Processor -- start 2024-03-28T17:59:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.276+0000 7f946d400640 1 -- start start 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94680cec50 0x7f94681e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 0x7f94680c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468162a10 con 0x7f94680b3210 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468162730 con 0x7f94680cec50 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.277+0000 7f946d400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94680d2fc0 con 0x7f9468155020 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46304/0 (socket says 172.21.15.62:46304) 2024-03-28T17:59:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 -- 172.21.15.62:0/2692219651 learned_addr learned my addr 172.21.15.62:0/2692219651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f946ca00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 0x7f94680c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94680cec50 0x7f94681e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 -- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 msgr2=0x7f94680c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 0x7f94680c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 -- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94680cec50 msgr2=0x7f94681e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94680cec50 0x7f94681e8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f9467e00640 1 -- 172.21.15.62:0/2692219651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94681e96e0 con 0x7f94680b3210 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.278+0000 7f946ca00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 0x7f94680c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f9467e00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9458000f80 tx=0x7f9458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458043020 con 0x7f94680b3210 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945802ce50 con 0x7f94680b3210 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458009510 con 0x7f94680b3210 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94681e99c0 con 0x7f94680b3210 2024-03-28T17:59:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.279+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94680b4a20 con 0x7f94680b3210 2024-03-28T17:59:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.280+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946804f400 con 0x7f94680b3210 2024-03-28T17:59:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.281+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9458046020 con 0x7f94680b3210 2024-03-28T17:59:37.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.283+0000 7f945fe00640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 0x7f944406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.283+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f94580ad160 con 0x7f94680b3210 2024-03-28T17:59:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.283+0000 7f9467400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 0x7f944406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.284+0000 7f9467400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 0x7f944406e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94681e7ce0 tx=0x7f9450008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.287+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f945804d030 con 0x7f94680b3210 2024-03-28T17:59:37.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.486+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9468067780 con 0x7f94680b3210 2024-03-28T17:59:37.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.487+0000 7f945fe00640 1 -- 172.21.15.62:0/2692219651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f945807bb40 con 0x7f94680b3210 2024-03-28T17:59:37.488 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 msgr2=0x7f944406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 0x7f944406e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94681e7ce0 tx=0x7f9450008040 comp rx=0 tx=0).stop 2024-03-28T17:59:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 msgr2=0x7f9468154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9458000f80 tx=0x7f9458004a40 comp rx=0 tx=0).stop 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 shutdown_connections 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944406bc00 0x7f944406e0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9468155020 0x7f94680c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94680cec50 0x7f94681e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 --2- 172.21.15.62:0/2692219651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94680b3210 0x7f9468154a20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 >> 172.21.15.62:0/2692219651 conn(0x7f9468066fb0 msgr2=0x7f94680b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 shutdown_connections 2024-03-28T17:59:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.490+0000 7f946d400640 1 -- 172.21.15.62:0/2692219651 wait complete. 2024-03-28T17:59:37.511 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.738+0000 7fac05600640 1 Processor -- start 2024-03-28T17:59:37.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.739+0000 7fac05600640 1 -- start start 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fac05600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac000b4c00 0x7fac00160190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fac05600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0015bec0 con 0x7fac000aee00 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fac05600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac000b1bf0 con 0x7fac000b15c0 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fac05600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000b1ed0 con 0x7fac000b4c00 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fabffe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fac000aee00 0x7fac0015ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41864/0 (socket says 172.21.15.62:41864) 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fabffe00640 1 -- 172.21.15.62:0/1537289077 learned_addr learned my addr 172.21.15.62:0/1537289077 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.741+0000 7fac04c00640 1 --2- 172.21.15.62:0/1537289077 >> v2:172.21.15.62:3300/0 conn(0x7fac000b4c00 0x7fac00160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.742+0000 7fabff400640 1 -- 172.21.15.62:0/1537289077 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469807237 0 0) 0x7fac0015bec0 con 0x7fac000aee00 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.742+0000 7fabff400640 1 -- 172.21.15.62:0/1537289077 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe4003520 con 0x7fac000aee00 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.742+0000 7fac04c00640 1 -- 172.21.15.62:0/1537289077 >> v1:172.21.15.81:6789/0 conn(0x7fac000b15c0 legacy=0x7fac000b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.742+0000 7fac04c00640 1 -- 172.21.15.62:0/1537289077 >> v1:172.21.15.62:6789/0 conn(0x7fac000aee00 legacy=0x7fac0015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.742+0000 7fac04c00640 1 -- 172.21.15.62:0/1537289077 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0015f6b0 con 0x7fac000b4c00 2024-03-28T17:59:37.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.743+0000 7fac04c00640 1 --2- 172.21.15.62:0/1537289077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b4c00 0x7fac00160190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009d90 tx=0x7fabf802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff35b924437e60e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.743+0000 7fabff400640 1 -- 172.21.15.62:0/1537289077 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8039070 con 0x7fac000b4c00 2024-03-28T17:59:37.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.743+0000 7fabff400640 1 -- 172.21.15.62:0/1537289077 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf802ccf0 con 0x7fac000b4c00 2024-03-28T17:59:37.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.743+0000 7fabff400640 1 -- 172.21.15.62:0/1537289077 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8034660 con 0x7fac000b4c00 2024-03-28T17:59:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 -- 172.21.15.62:0/1537289077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b4c00 msgr2=0x7fac00160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 --2- 172.21.15.62:0/1537289077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b4c00 0x7fac00160190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009d90 tx=0x7fabf802bf40 comp rx=0 tx=0).stop 2024-03-28T17:59:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 -- 172.21.15.62:0/1537289077 shutdown_connections 2024-03-28T17:59:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 --2- 172.21.15.62:0/1537289077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b4c00 0x7fac00160190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 -- 172.21.15.62:0/1537289077 >> 172.21.15.62:0/1537289077 conn(0x7fac00066dd0 msgr2=0x7fac000671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 -- 172.21.15.62:0/1537289077 shutdown_connections 2024-03-28T17:59:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.744+0000 7fac05600640 1 -- 172.21.15.62:0/1537289077 wait complete. 2024-03-28T17:59:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.746+0000 7fac05600640 1 Processor -- start 2024-03-28T17:59:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.746+0000 7fac05600640 1 -- start start 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac000aee00 0x7fac00157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 0x7fac001460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000af1e0 con 0x7fac000b15c0 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0015fc70 con 0x7fac000b4c00 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fac05600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000b2930 con 0x7fac000aee00 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46312/0 (socket says 172.21.15.62:46312) 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.747+0000 7fabf7e00640 1 -- 172.21.15.62:0/2406675980 learned_addr learned my addr 172.21.15.62:0/2406675980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabffe00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac000aee00 0x7fac00157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fac04c00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 0x7fac001460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 -- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac000aee00 msgr2=0x7fac00157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac000aee00 0x7fac00157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 -- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 msgr2=0x7fac001460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 0x7fac001460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 -- 172.21.15.62:0/2406675980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0015eb30 con 0x7fac000b15c0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fac04c00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 0x7fac001460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabf7e00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c400 tx=0x7fabe800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80052d0 con 0x7fac000b15c0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8005490 con 0x7fac000b15c0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0015dfb0 con 0x7fac000b15c0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.748+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8007260 con 0x7fac000b15c0 2024-03-28T17:59:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.749+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0015e570 con 0x7fac000b15c0 2024-03-28T17:59:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.750+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0004f400 con 0x7fac000b15c0 2024-03-28T17:59:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.751+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fabe8007400 con 0x7fac000b15c0 2024-03-28T17:59:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.752+0000 7fabfcc00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 0x7fabdc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.752+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fabe808a100 con 0x7fac000b15c0 2024-03-28T17:59:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.752+0000 7fabffe00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 0x7fabdc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.754+0000 7fabffe00640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 0x7fabdc06e090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabf00011e0 tx=0x7fabf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.757+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fabe8058ad0 con 0x7fac000b15c0 2024-03-28T17:59:37.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.961+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac00067740 con 0x7fac000b15c0 2024-03-28T17:59:37.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.962+0000 7fabfcc00640 1 -- 172.21.15.62:0/2406675980 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabe80583d0 con 0x7fac000b15c0 2024-03-28T17:59:37.962 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:37.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 msgr2=0x7fabdc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 0x7fabdc06e090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabf00011e0 tx=0x7fabf0009040 comp rx=0 tx=0).stop 2024-03-28T17:59:37.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 msgr2=0x7fac00153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c400 tx=0x7fabe800c8c0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 shutdown_connections 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fabdc06bbd0 0x7fabdc06e090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000b4c00 0x7fac001460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac000b15c0 0x7fac00153ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 --2- 172.21.15.62:0/2406675980 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fac000aee00 0x7fac00157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 >> 172.21.15.62:0/2406675980 conn(0x7fac00066dd0 msgr2=0x7fac0015b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 shutdown_connections 2024-03-28T17:59:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:37.965+0000 7fac05600640 1 -- 172.21.15.62:0/2406675980 wait complete. 2024-03-28T17:59:37.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-03-28T17:59:38.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.207+0000 7fa83f200640 1 Processor -- start 2024-03-28T17:59:38.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.207+0000 7fa83f200640 1 -- start start 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa84014bb00 0x7fa840072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 0x7fa8400b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400b4aa0 con 0x7fa8400b3c60 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8400b4d80 con 0x7fa84014bb00 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa840070b70 con 0x7fa84014bee0 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa8400b3c60 0x7fa84006caa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41876/0 (socket says 172.21.15.62:41876) 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83e800640 1 -- 172.21.15.62:0/3584163739 learned_addr learned my addr 172.21.15.62:0/3584163739 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa83de00640 1 --2- 172.21.15.62:0/3584163739 >> v2:172.21.15.62:3300/0 conn(0x7fa84014bb00 0x7fa840072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.209+0000 7fa837e00640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 0x7fa8400b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83d400640 1 -- 172.21.15.62:0/3584163739 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815918531 0 0) 0x7fa8400b4aa0 con 0x7fa8400b3c60 2024-03-28T17:59:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83d400640 1 -- 172.21.15.62:0/3584163739 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa824003520 con 0x7fa8400b3c60 2024-03-28T17:59:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83de00640 1 -- 172.21.15.62:0/3584163739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 msgr2=0x7fa8400b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83de00640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 0x7fa8400b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83de00640 1 -- 172.21.15.62:0/3584163739 >> v1:172.21.15.62:6789/0 conn(0x7fa8400b3c60 legacy=0x7fa84006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa83de00640 1 -- 172.21.15.62:0/3584163739 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840070e50 con 0x7fa84014bb00 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.210+0000 7fa837e00640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 0x7fa8400b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.211+0000 7fa83de00640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84014bb00 0x7fa840072690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097a0 tx=0x7fa83002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8400071e2ea948 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83d400640 1 -- 172.21.15.62:0/3584163739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830031780 con 0x7fa84014bb00 2024-03-28T17:59:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83d400640 1 -- 172.21.15.62:0/3584163739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830037070 con 0x7fa84014bb00 2024-03-28T17:59:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83d400640 1 -- 172.21.15.62:0/3584163739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830031b40 con 0x7fa84014bb00 2024-03-28T17:59:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 -- 172.21.15.62:0/3584163739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84014bb00 msgr2=0x7fa840072690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84014bb00 0x7fa840072690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097a0 tx=0x7fa83002b820 comp rx=0 tx=0).stop 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 -- 172.21.15.62:0/3584163739 shutdown_connections 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bee0 0x7fa8400b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 --2- 172.21.15.62:0/3584163739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84014bb00 0x7fa840072690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 -- 172.21.15.62:0/3584163739 >> 172.21.15.62:0/3584163739 conn(0x7fa8400b1730 msgr2=0x7fa8400b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.212+0000 7fa83f200640 1 -- 172.21.15.62:0/3584163739 shutdown_connections 2024-03-28T17:59:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.213+0000 7fa83f200640 1 -- 172.21.15.62:0/3584163739 wait complete. 2024-03-28T17:59:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.214+0000 7fa83f200640 1 Processor -- start 2024-03-28T17:59:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.214+0000 7fa83f200640 1 -- start start 2024-03-28T17:59:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bb00 0x7fa84013ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa84014bee0 0x7fa8401449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400b46e0 con 0x7fa8400b3c60 2024-03-28T17:59:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840071130 con 0x7fa84014bee0 2024-03-28T17:59:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400b4860 con 0x7fa84014bb00 2024-03-28T17:59:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46330/0 (socket says 172.21.15.62:46330) 2024-03-28T17:59:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.215+0000 7fa83de00640 1 -- 172.21.15.62:0/1063415515 learned_addr learned my addr 172.21.15.62:0/1063415515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa837e00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bb00 0x7fa84013ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83e800640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa84014bee0 0x7fa8401449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 -- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bb00 msgr2=0x7fa84013ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bb00 0x7fa84013ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 -- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa84014bee0 msgr2=0x7fa8401449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa84014bee0 0x7fa8401449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 -- 172.21.15.62:0/1063415515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840143620 con 0x7fa8400b3c60 2024-03-28T17:59:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa83de00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098c0 tx=0x7fa8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830032610 con 0x7fa8400b3c60 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8300327b0 con 0x7fa8400b3c60 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.216+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830032ba0 con 0x7fa8400b3c60 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.217+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa840141930 con 0x7fa8400b3c60 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.217+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa840141ef0 con 0x7fa8400b3c60 2024-03-28T17:59:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.218+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa840073490 con 0x7fa8400b3c60 2024-03-28T17:59:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.219+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa83004e440 con 0x7fa8400b3c60 2024-03-28T17:59:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.220+0000 7fa836000640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 0x7fa81c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.220+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa8300ac8f0 con 0x7fa8400b3c60 2024-03-28T17:59:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.220+0000 7fa837e00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 0x7fa81c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.222+0000 7fa837e00640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 0x7fa81c06e090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa84015c8a0 tx=0x7fa828008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:38.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.224+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa830040070 con 0x7fa8400b3c60 2024-03-28T17:59:38.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:38.490+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7fa84004f4e0 con 0x7fa8400b3c60 2024-03-28T17:59:39.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.265+0000 7fa836000640 1 -- 172.21.15.62:0/1063415515 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v64) v1 ==== 147+0+0 (secure 0 0 0) 0x7fa83002f050 con 0x7fa8400b3c60 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.269+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 msgr2=0x7fa81c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.269+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 0x7fa81c06e090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa84015c8a0 tx=0x7fa828008040 comp rx=0 tx=0).stop 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 msgr2=0x7fa840164010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098c0 tx=0x7fa8300046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 shutdown_connections 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa81c06bbd0 0x7fa81c06e090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa84014bee0 0x7fa8401449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa84014bb00 0x7fa84013ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 --2- 172.21.15.62:0/1063415515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8400b3c60 0x7fa840164010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 >> 172.21.15.62:0/1063415515 conn(0x7fa8400b1730 msgr2=0x7fa840163900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 shutdown_connections 2024-03-28T17:59:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.270+0000 7fa83f200640 1 -- 172.21.15.62:0/1063415515 wait complete. 2024-03-28T17:59:39.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T17:59:39.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.514+0000 7faff8a00640 1 Processor -- start 2024-03-28T17:59:39.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.514+0000 7faff8a00640 1 -- start start 2024-03-28T17:59:39.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.515+0000 7faff8a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faff415e2c0 0x7faff40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.515+0000 7faff8a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff40b3210 0x7faff40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.515+0000 7faff8a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff4161bb0 con 0x7faff40cec50 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.515+0000 7faff8a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff4161e90 con 0x7faff415e2c0 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.515+0000 7faff8a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4162170 con 0x7faff40b3210 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faff415e2c0 0x7faff40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faff415e2c0 0x7faff40ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34200/0 (socket says 172.21.15.62:34200) 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 -- 172.21.15.62:0/2182104089 learned_addr learned my addr 172.21.15.62:0/2182104089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff3200640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff40b3210 0x7faff40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 -- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff40b3210 msgr2=0x7faff40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:39.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff40b3210 0x7faff40d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 -- 172.21.15.62:0/2182104089 >> v1:172.21.15.81:6789/0 conn(0x7faff40cec50 legacy=0x7faff40cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.516+0000 7faff1e00640 1 -- 172.21.15.62:0/2182104089 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4162450 con 0x7faff415e2c0 2024-03-28T17:59:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.518+0000 7faff1e00640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff415e2c0 0x7faff40ce570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009870 tx=0x7fafdc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3714730d62c9e633 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.518+0000 7faff1400640 1 -- 172.21.15.62:0/2182104089 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc034070 con 0x7faff415e2c0 2024-03-28T17:59:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.518+0000 7faff1400640 1 -- 172.21.15.62:0/2182104089 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc030840 con 0x7faff415e2c0 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.518+0000 7faff1400640 1 -- 172.21.15.62:0/2182104089 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc030c10 con 0x7faff415e2c0 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 -- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff415e2c0 msgr2=0x7faff40ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff415e2c0 0x7faff40ce570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009870 tx=0x7fafdc02b980 comp rx=0 tx=0).stop 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 -- 172.21.15.62:0/2182104089 shutdown_connections 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff40b3210 0x7faff40d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 --2- 172.21.15.62:0/2182104089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff415e2c0 0x7faff40ce570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 -- 172.21.15.62:0/2182104089 >> 172.21.15.62:0/2182104089 conn(0x7faff4066fb0 msgr2=0x7faff40673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 -- 172.21.15.62:0/2182104089 shutdown_connections 2024-03-28T17:59:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.519+0000 7faff8a00640 1 -- 172.21.15.62:0/2182104089 wait complete. 2024-03-28T17:59:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.521+0000 7faff8a00640 1 Processor -- start 2024-03-28T17:59:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.521+0000 7faff8a00640 1 -- start start 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff40b3210 0x7faff40c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 0x7faff415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40d24f0 con 0x7faff40cec50 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4162a10 con 0x7faff40b3210 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff8a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40d2670 con 0x7faff4146f70 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46334/0 (socket says 172.21.15.62:46334) 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.522+0000 7faff1e00640 1 -- 172.21.15.62:0/2245676935 learned_addr learned my addr 172.21.15.62:0/2245676935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff2800640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff40b3210 0x7faff40c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff3200640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 0x7faff415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 -- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 msgr2=0x7faff415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 0x7faff415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 -- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff40b3210 msgr2=0x7faff40c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff40b3210 0x7faff40c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 -- 172.21.15.62:0/2245676935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40b5a30 con 0x7faff40cec50 2024-03-28T17:59:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff3200640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 0x7faff415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff1e00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafdc02b9c0 tx=0x7fafdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc034040 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc03ada0 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff40b5d10 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.523+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc044610 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.524+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff40b62d0 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.525+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff40b6510 con 0x7faff40cec50 2024-03-28T17:59:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.526+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fafdc04b060 con 0x7faff40cec50 2024-03-28T17:59:39.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.526+0000 7fafeaa00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 0x7fafd006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:39.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.527+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fafdc0acb20 con 0x7faff40cec50 2024-03-28T17:59:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.527+0000 7faff2800640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 0x7fafd006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.528+0000 7faff2800640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 0x7fafd006e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000f60 tx=0x7fafe4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.531+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fafdc03f070 con 0x7faff40cec50 2024-03-28T17:59:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.730+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faff404f400 con 0x7faff40cec50 2024-03-28T17:59:39.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.737+0000 7fafeaa00640 1 -- 172.21.15.62:0/2245676935 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fafdc07b4f0 con 0x7faff40cec50 2024-03-28T17:59:39.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.739+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 msgr2=0x7fafd006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.739+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 0x7fafd006e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000f60 tx=0x7fafe4009040 comp rx=0 tx=0).stop 2024-03-28T17:59:39.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.739+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 msgr2=0x7faff40b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.739+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafdc02b9c0 tx=0x7fafdc0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 shutdown_connections 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fafd006bbd0 0x7fafd006e090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faff4146f70 0x7faff415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff40cec50 0x7faff40b50c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 --2- 172.21.15.62:0/2245676935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff40b3210 0x7faff40c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 >> 172.21.15.62:0/2245676935 conn(0x7faff4066fb0 msgr2=0x7faff415b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 shutdown_connections 2024-03-28T17:59:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:39.740+0000 7faff8a00640 1 -- 172.21.15.62:0/2245676935 wait complete. 2024-03-28T17:59:39.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T17:59:39.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T17:59:39.878 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T17:59:40.007 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T17:59:40.007 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:40.007 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:40.067 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:40.067 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:40.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:40.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.338+0000 7f8087e00640 1 Processor -- start 2024-03-28T17:59:40.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.338+0000 7f8087e00640 1 -- start start 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8088160090 0x7f8088070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80880cebb0 con 0x7f8088072ab0 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80880cee90 con 0x7f80881456d0 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80880cf170 con 0x7f8088160090 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8088072ab0 0x7f8088070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41884/0 (socket says 172.21.15.62:41884) 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f8087400640 1 -- 172.21.15.62:0/156488560 learned_addr learned my addr 172.21.15.62:0/156488560 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.340+0000 7f807fe00640 1 --2- 172.21.15.62:0/156488560 >> v2:172.21.15.62:3300/0 conn(0x7f8088160090 0x7f8088070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.341+0000 7f807fe00640 1 -- 172.21.15.62:0/156488560 >> v1:172.21.15.81:6789/0 conn(0x7f80881456d0 legacy=0x7f808815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:40.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.341+0000 7f807fe00640 1 -- 172.21.15.62:0/156488560 >> v1:172.21.15.62:6789/0 conn(0x7f8088072ab0 legacy=0x7f8088070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.341+0000 7f807fe00640 1 -- 172.21.15.62:0/156488560 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880cf450 con 0x7f8088160090 2024-03-28T17:59:40.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.341+0000 7f8086000640 1 -- 172.21.15.62:0/156488560 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071191033 0 0) 0x7f80880cebb0 con 0x7f8088072ab0 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.342+0000 7f807fe00640 1 --2- 172.21.15.62:0/156488560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088070a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8070009790 tx=0x7f807002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4c2ec669f6e09f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.342+0000 7f8086000640 1 -- 172.21.15.62:0/156488560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070035020 con 0x7f8088160090 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.342+0000 7f8086000640 1 -- 172.21.15.62:0/156488560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070031810 con 0x7f8088160090 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.342+0000 7f8086000640 1 -- 172.21.15.62:0/156488560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070031be0 con 0x7f8088160090 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 -- 172.21.15.62:0/156488560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 msgr2=0x7f8088070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 --2- 172.21.15.62:0/156488560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088070a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8070009790 tx=0x7f807002b8b0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 -- 172.21.15.62:0/156488560 shutdown_connections 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 --2- 172.21.15.62:0/156488560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088070a40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 -- 172.21.15.62:0/156488560 >> 172.21.15.62:0/156488560 conn(0x7f80880b1730 msgr2=0x7f80880b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 -- 172.21.15.62:0/156488560 shutdown_connections 2024-03-28T17:59:40.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.343+0000 7f8087e00640 1 -- 172.21.15.62:0/156488560 wait complete. 2024-03-28T17:59:40.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.344+0000 7f8087e00640 1 Processor -- start 2024-03-28T17:59:40.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.345+0000 7f8087e00640 1 -- start start 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.345+0000 7f8087e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 0x7f808813bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.345+0000 7f8087e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80881456d0 0x7f8088139100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.345+0000 7f8087e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808806dd90 con 0x7f8088160090 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80880cf730 con 0x7f8088072ab0 2024-03-28T17:59:40.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808806df10 con 0x7f80881456d0 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46348/0 (socket says 172.21.15.62:46348) 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 -- 172.21.15.62:0/4242928620 learned_addr learned my addr 172.21.15.62:0/4242928620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8086a00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 0x7f808813bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 -- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80881456d0 msgr2=0x7f8088139100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80881456d0 0x7f8088139100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 -- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 msgr2=0x7f808813bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 0x7f808813bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8087400640 1 -- 172.21.15.62:0/4242928620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880b3490 con 0x7f8088160090 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.346+0000 7f8086a00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 0x7f808813bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f8087400640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7d0 tx=0x7f808000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080018070 con 0x7f8088160090 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000a640 con 0x7f8088160090 2024-03-28T17:59:40.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000aa70 con 0x7f8088160090 2024-03-28T17:59:40.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80880b3730 con 0x7f8088160090 2024-03-28T17:59:40.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.347+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808815acb0 con 0x7f8088160090 2024-03-28T17:59:40.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.348+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808804f4e0 con 0x7f8088160090 2024-03-28T17:59:40.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.349+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f808001b070 con 0x7f8088160090 2024-03-28T17:59:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.350+0000 7f807ea00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 0x7f806406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.350+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f808008a220 con 0x7f8088160090 2024-03-28T17:59:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.350+0000 7f8086a00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 0x7f806406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.351+0000 7f8086a00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 0x7f806406e090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.355+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8080058b60 con 0x7f8088160090 2024-03-28T17:59:40.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.563+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f808806e6e0 con 0x7f8088160090 2024-03-28T17:59:40.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.566+0000 7f807ea00640 1 -- 172.21.15.62:0/4242928620 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f8080058460 con 0x7f8088160090 2024-03-28T17:59:40.568 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:40.568 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 msgr2=0x7f806406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 0x7f806406e090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078009040 comp rx=0 tx=0).stop 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 msgr2=0x7f8088159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7d0 tx=0x7f808000cc90 comp rx=0 tx=0).stop 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 shutdown_connections 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f806406bbd0 0x7f806406e090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8088160090 0x7f8088159b60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f80881456d0 0x7f8088139100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 --2- 172.21.15.62:0/4242928620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8088072ab0 0x7f808813bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.569+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 >> 172.21.15.62:0/4242928620 conn(0x7f80880b1730 msgr2=0x7f808815b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:40.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.570+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 shutdown_connections 2024-03-28T17:59:40.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.570+0000 7f8087e00640 1 -- 172.21.15.62:0/4242928620 wait complete. 2024-03-28T17:59:40.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:40.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:40.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.807+0000 7fb097200640 1 Processor -- start 2024-03-28T17:59:40.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.807+0000 7fb097200640 1 -- start start 2024-03-28T17:59:40.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb097200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0900b15c0 0x7fb0900b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb097200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb090070890 0x7fb0900723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb097200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900b1ee0 con 0x7fb090072900 2024-03-28T17:59:40.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb097200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0901628a0 con 0x7fb090070890 2024-03-28T17:59:40.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb097200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090162b80 con 0x7fb0900b15c0 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb095400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb090070890 0x7fb0900723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb095400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb090070890 0x7fb0900723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46362/0 (socket says 172.21.15.62:46362) 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb095400640 1 -- 172.21.15.62:0/2939225328 learned_addr learned my addr 172.21.15.62:0/2939225328 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.809+0000 7fb095e00640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0900b15c0 0x7fb0900b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.810+0000 7fb095400640 1 -- 172.21.15.62:0/2939225328 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0900b15c0 msgr2=0x7fb0900b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.810+0000 7fb095400640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0900b15c0 0x7fb0900b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.810+0000 7fb095400640 1 -- 172.21.15.62:0/2939225328 >> v1:172.21.15.81:6789/0 conn(0x7fb090072900 legacy=0x7fb09006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.810+0000 7fb095400640 1 -- 172.21.15.62:0/2939225328 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090162e60 con 0x7fb090070890 2024-03-28T17:59:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.811+0000 7fb095400640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0900723c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb080009870 tx=0x7fb08002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da1a346c747ee2a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.811+0000 7fb094a00640 1 -- 172.21.15.62:0/2939225328 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080034070 con 0x7fb090070890 2024-03-28T17:59:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.811+0000 7fb094a00640 1 -- 172.21.15.62:0/2939225328 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080030840 con 0x7fb090070890 2024-03-28T17:59:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.811+0000 7fb094a00640 1 -- 172.21.15.62:0/2939225328 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080030c10 con 0x7fb090070890 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 -- 172.21.15.62:0/2939225328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 msgr2=0x7fb0900723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0900723c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb080009870 tx=0x7fb08002b980 comp rx=0 tx=0).stop 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 -- 172.21.15.62:0/2939225328 shutdown_connections 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0900723c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 --2- 172.21.15.62:0/2939225328 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb0900b15c0 0x7fb0900b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 -- 172.21.15.62:0/2939225328 >> 172.21.15.62:0/2939225328 conn(0x7fb090066dd0 msgr2=0x7fb0900671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:40.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 -- 172.21.15.62:0/2939225328 shutdown_connections 2024-03-28T17:59:40.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.812+0000 7fb097200640 1 -- 172.21.15.62:0/2939225328 wait complete. 2024-03-28T17:59:40.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.814+0000 7fb097200640 1 Processor -- start 2024-03-28T17:59:40.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.814+0000 7fb097200640 1 -- start start 2024-03-28T17:59:40.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb090072900 0x7fb09015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 0x7fb09015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900b2b50 con 0x7fb090070890 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090163420 con 0x7fb0900b15c0 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb097200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900af130 con 0x7fb090072900 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb095e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb095e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46372/0 (socket says 172.21.15.62:46372) 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.815+0000 7fb095e00640 1 -- 172.21.15.62:0/3603488219 learned_addr learned my addr 172.21.15.62:0/3603488219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095400640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb090072900 0x7fb09015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb096800640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 0x7fb09015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 -- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb090072900 msgr2=0x7fb09015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb090072900 0x7fb09015b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 -- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 msgr2=0x7fb09015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 0x7fb09015b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 -- 172.21.15.62:0/3603488219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0901ed720 con 0x7fb090070890 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb096800640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 0x7fb09015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb095e00640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb088007ec0 tx=0x7fb08800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088014620 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088019070 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0901eda00 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088014a20 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.816+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901edfc0 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.817+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09004f400 con 0x7fb090070890 2024-03-28T17:59:40.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.818+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb088014bc0 con 0x7fb090070890 2024-03-28T17:59:40.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.820+0000 7fb086000640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 0x7fb07406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:40.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.820+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb08808b500 con 0x7fb090070890 2024-03-28T17:59:40.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.820+0000 7fb095400640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 0x7fb07406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:40.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.821+0000 7fb095400640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 0x7fb07406e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb080036000 tx=0x7fb08003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:40.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:40.825+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb088059ed0 con 0x7fb090070890 2024-03-28T17:59:41.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.033+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb09006e480 con 0x7fb090070890 2024-03-28T17:59:41.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.034+0000 7fb086000640 1 -- 172.21.15.62:0/3603488219 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0880597d0 con 0x7fb090070890 2024-03-28T17:59:41.034 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 msgr2=0x7fb07406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 0x7fb07406e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb080036000 tx=0x7fb08003b040 comp rx=0 tx=0).stop 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 msgr2=0x7fb0901580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb088007ec0 tx=0x7fb08800e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 shutdown_connections 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb07406bc00 0x7fb07406e0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0900b15c0 0x7fb09015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb090072900 0x7fb09015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 --2- 172.21.15.62:0/3603488219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb090070890 0x7fb0901580f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 >> 172.21.15.62:0/3603488219 conn(0x7fb090066dd0 msgr2=0x7fb0900cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:41.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 shutdown_connections 2024-03-28T17:59:41.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.037+0000 7fb097200640 1 -- 172.21.15.62:0/3603488219 wait complete. 2024-03-28T17:59:41.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:41.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.273+0000 7f200ee00640 1 Processor -- start 2024-03-28T17:59:41.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.273+0000 7f200ee00640 1 -- start start 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ee00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2008071dd0 0x7f20080d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200806cdd0 con 0x7f20080b6fb0 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ee00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080d1af0 con 0x7f200815e020 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ee00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080d1dd0 con 0x7f2008071dd0 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ce00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f20080b6fb0 0x7f20080d30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41894/0 (socket says 172.21.15.62:41894) 2024-03-28T17:59:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f200ce00640 1 -- 172.21.15.62:0/2909134131 learned_addr learned my addr 172.21.15.62:0/2909134131 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.275+0000 7f2007400640 1 --2- 172.21.15.62:0/2909134131 >> v2:172.21.15.81:3300/0 conn(0x7f2008071dd0 0x7f20080d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.276+0000 7f2006a00640 1 -- 172.21.15.62:0/2909134131 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471714997 0 0) 0x7f200806cdd0 con 0x7f20080b6fb0 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.276+0000 7f2006a00640 1 -- 172.21.15.62:0/2909134131 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fec003520 con 0x7f20080b6fb0 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.276+0000 7f2007400640 1 -- 172.21.15.62:0/2909134131 >> v1:172.21.15.81:6789/0 conn(0x7f200815e020 legacy=0x7f20080716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.276+0000 7f2007400640 1 -- 172.21.15.62:0/2909134131 >> v1:172.21.15.62:6789/0 conn(0x7f20080b6fb0 legacy=0x7f20080d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.276+0000 7f2007400640 1 -- 172.21.15.62:0/2909134131 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080d20b0 con 0x7f2008071dd0 2024-03-28T17:59:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.278+0000 7f2007400640 1 --2- 172.21.15.62:0/2909134131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f20080d2b90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097a0 tx=0x7f1ff402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26703f3e8d016c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.278+0000 7f2006a00640 1 -- 172.21.15.62:0/2909134131 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4034070 con 0x7f2008071dd0 2024-03-28T17:59:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.278+0000 7f2006a00640 1 -- 172.21.15.62:0/2909134131 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4030610 con 0x7f2008071dd0 2024-03-28T17:59:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.278+0000 7f2006a00640 1 -- 172.21.15.62:0/2909134131 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40309e0 con 0x7f2008071dd0 2024-03-28T17:59:41.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 -- 172.21.15.62:0/2909134131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 msgr2=0x7f20080d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 --2- 172.21.15.62:0/2909134131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f20080d2b90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097a0 tx=0x7f1ff402b960 comp rx=0 tx=0).stop 2024-03-28T17:59:41.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 -- 172.21.15.62:0/2909134131 shutdown_connections 2024-03-28T17:59:41.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 --2- 172.21.15.62:0/2909134131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f20080d2b90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 -- 172.21.15.62:0/2909134131 >> 172.21.15.62:0/2909134131 conn(0x7f20080d3d80 msgr2=0x7f20080d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:41.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 -- 172.21.15.62:0/2909134131 shutdown_connections 2024-03-28T17:59:41.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.279+0000 7f200ee00640 1 -- 172.21.15.62:0/2909134131 wait complete. 2024-03-28T17:59:41.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.281+0000 7f200ee00640 1 Processor -- start 2024-03-28T17:59:41.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.281+0000 7f200ee00640 1 -- start start 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f2008152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 0x7f20081f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200815e400 con 0x7f200815e020 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080d2670 con 0x7f2008071dd0 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ee00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200815e580 con 0x7f20080b6fb0 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46382/0 (socket says 172.21.15.62:46382) 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.282+0000 7f200ce00640 1 -- 172.21.15.62:0/731007118 learned_addr learned my addr 172.21.15.62:0/731007118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f2007e00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f2008152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f2007400640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 0x7f20081f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 -- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 msgr2=0x7f20081f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 0x7f20081f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 -- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 msgr2=0x7f2008152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f2008152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 -- 172.21.15.62:0/731007118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20081f5350 con 0x7f200815e020 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f2007400640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 0x7f20081f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ce00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f200000cb40 tx=0x7f200000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000017070 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200000aca0 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000012620 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.283+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20081f5630 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.284+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20081f5bf0 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.285+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20080680c0 con 0x7f200815e020 2024-03-28T17:59:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.286+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f200001a070 con 0x7f200815e020 2024-03-28T17:59:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.286+0000 7f1fffe00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 0x7f1fe806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.287+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f200008a8c0 con 0x7f200815e020 2024-03-28T17:59:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.287+0000 7f2007e00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 0x7f1fe806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.288+0000 7f2007e00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 0x7f1fe806e090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000f60 tx=0x7f1ff8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.292+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f20000593c0 con 0x7f200815e020 2024-03-28T17:59:41.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.493+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20080728e0 con 0x7f200815e020 2024-03-28T17:59:41.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.494+0000 7f1fffe00640 1 -- 172.21.15.62:0/731007118 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f200005f0c0 con 0x7f200815e020 2024-03-28T17:59:41.494 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 msgr2=0x7f1fe806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 0x7f1fe806e090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000f60 tx=0x7f1ff8009040 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 msgr2=0x7f20081f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f200000cb40 tx=0x7f200000a060 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 shutdown_connections 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1fe806bbd0 0x7f1fe806e090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f200815e020 0x7f20081f63f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f20080b6fb0 0x7f20081f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 --2- 172.21.15.62:0/731007118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2008071dd0 0x7f2008152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.497+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 >> 172.21.15.62:0/731007118 conn(0x7f20080d3d80 msgr2=0x7f20080b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.498+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 shutdown_connections 2024-03-28T17:59:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.498+0000 7f200ee00640 1 -- 172.21.15.62:0/731007118 wait complete. 2024-03-28T17:59:41.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:41.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.730+0000 7fd8dee00640 1 Processor -- start 2024-03-28T17:59:41.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.731+0000 7fd8dee00640 1 -- start start 2024-03-28T17:59:41.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.732+0000 7fd8dee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8d815e2c0 0x7fd8d80ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d8161bb0 con 0x7fd8d80b3210 2024-03-28T17:59:41.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dee00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d8161e90 con 0x7fd8d80cec50 2024-03-28T17:59:41.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d8162170 con 0x7fd8d815e2c0 2024-03-28T17:59:41.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8d815e2c0 0x7fd8d80ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8d815e2c0 0x7fd8d80ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46398/0 (socket says 172.21.15.62:46398) 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8dd000640 1 -- 172.21.15.62:0/426490709 learned_addr learned my addr 172.21.15.62:0/426490709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.733+0000 7fd8de400640 1 --1- 172.21.15.62:0/426490709 >> v1:172.21.15.62:6789/0 conn(0x7fd8d80b3210 0x7fd8d80d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41910/0 (socket says 172.21.15.62:41910) 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.734+0000 7fd8dd000640 1 -- 172.21.15.62:0/426490709 >> v1:172.21.15.81:6789/0 conn(0x7fd8d80cec50 legacy=0x7fd8d80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.734+0000 7fd8dd000640 1 -- 172.21.15.62:0/426490709 >> v1:172.21.15.62:6789/0 conn(0x7fd8d80b3210 legacy=0x7fd8d80d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.734+0000 7fd8dd000640 1 -- 172.21.15.62:0/426490709 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d8162450 con 0x7fd8d815e2c0 2024-03-28T17:59:41.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.734+0000 7fd8d7e00640 1 -- 172.21.15.62:0/426490709 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531555261 0 0) 0x7fd8d8161bb0 con 0x7fd8d80b3210 2024-03-28T17:59:41.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.735+0000 7fd8dd000640 1 --2- 172.21.15.62:0/426490709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d815e2c0 0x7fd8d80ce570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c402ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a665ae7ca1b0dd5c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.735+0000 7fd8d7e00640 1 -- 172.21.15.62:0/426490709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c402f870 con 0x7fd8d815e2c0 2024-03-28T17:59:41.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.735+0000 7fd8d7e00640 1 -- 172.21.15.62:0/426490709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c402fa10 con 0x7fd8d815e2c0 2024-03-28T17:59:41.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.735+0000 7fd8d7e00640 1 -- 172.21.15.62:0/426490709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c402fde0 con 0x7fd8d815e2c0 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 -- 172.21.15.62:0/426490709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d815e2c0 msgr2=0x7fd8d80ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 --2- 172.21.15.62:0/426490709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d815e2c0 0x7fd8d80ce570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c402ba50 comp rx=0 tx=0).stop 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 -- 172.21.15.62:0/426490709 shutdown_connections 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 --2- 172.21.15.62:0/426490709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d815e2c0 0x7fd8d80ce570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 -- 172.21.15.62:0/426490709 >> 172.21.15.62:0/426490709 conn(0x7fd8d8066fb0 msgr2=0x7fd8d80673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 -- 172.21.15.62:0/426490709 shutdown_connections 2024-03-28T17:59:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.736+0000 7fd8dee00640 1 -- 172.21.15.62:0/426490709 wait complete. 2024-03-28T17:59:41.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.738+0000 7fd8dee00640 1 Processor -- start 2024-03-28T17:59:41.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.738+0000 7fd8dee00640 1 -- start start 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8d80cec50 0x7fd8d8161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d80c7720 0x7fd8d80b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80d26c0 con 0x7fd8d80b3210 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8162a10 con 0x7fd8d80c7720 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dee00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80d2840 con 0x7fd8d80cec50 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46410/0 (socket says 172.21.15.62:46410) 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.739+0000 7fd8dda00640 1 -- 172.21.15.62:0/163042356 learned_addr learned my addr 172.21.15.62:0/163042356 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8de400640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d80c7720 0x7fd8d80b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dd000640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8d80cec50 0x7fd8d8161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 -- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8d80cec50 msgr2=0x7fd8d8161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8d80cec50 0x7fd8d8161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 -- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d80c7720 msgr2=0x7fd8d80b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d80c7720 0x7fd8d80b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 -- 172.21.15.62:0/163042356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80c87a0 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8dda00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c450 tx=0x7fd8cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.740+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0052d0 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.741+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d80c7f00 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.741+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc005470 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.741+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc012870 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.741+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d80c84c0 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.742+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d804f400 con 0x7fd8d80b3210 2024-03-28T17:59:41.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.743+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd8cc012a10 con 0x7fd8d80b3210 2024-03-28T17:59:41.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.744+0000 7fd8d5600640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 0x7fd8b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.744+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd8cc08a730 con 0x7fd8d80b3210 2024-03-28T17:59:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.744+0000 7fd8dd000640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 0x7fd8b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:41.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.745+0000 7fd8dd000640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 0x7fd8b806e090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4036000 tx=0x7fd8c4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:41.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.748+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd8cc059190 con 0x7fd8d80b3210 2024-03-28T17:59:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.949+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8d80d23c0 con 0x7fd8d80b3210 2024-03-28T17:59:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.950+0000 7fd8d5600640 1 -- 172.21.15.62:0/163042356 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8cc058a90 con 0x7fd8d80b3210 2024-03-28T17:59:41.951 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 msgr2=0x7fd8b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 0x7fd8b806e090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4036000 tx=0x7fd8c4008030 comp rx=0 tx=0).stop 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 msgr2=0x7fd8d815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c450 tx=0x7fd8cc00c910 comp rx=0 tx=0).stop 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 shutdown_connections 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8b806bbd0 0x7fd8b806e090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d80c7720 0x7fd8d80b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8d80cec50 0x7fd8d8161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 --2- 172.21.15.62:0/163042356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d80b3210 0x7fd8d815e6b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:41.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 >> 172.21.15.62:0/163042356 conn(0x7fd8d8066fb0 msgr2=0x7fd8d8067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:41.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 shutdown_connections 2024-03-28T17:59:41.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:41.953+0000 7fd8dee00640 1 -- 172.21.15.62:0/163042356 wait complete. 2024-03-28T17:59:41.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:42.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.180+0000 7faa4cc00640 1 Processor -- start 2024-03-28T17:59:42.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.180+0000 7faa4cc00640 1 -- start start 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.181+0000 7faa4cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa480cf590 0x7faa480b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.181+0000 7faa4cc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa48160110 con 0x7faa48152f90 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.181+0000 7faa4cc00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa481603c0 con 0x7faa4806ccb0 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.181+0000 7faa4cc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa481606a0 con 0x7faa480cf590 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa480cf590 0x7faa480b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa480cf590 0x7faa480b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46418/0 (socket says 172.21.15.62:46418) 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 -- 172.21.15.62:0/582866174 learned_addr learned my addr 172.21.15.62:0/582866174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa45e00640 1 --1- 172.21.15.62:0/582866174 >> v1:172.21.15.62:6789/0 conn(0x7faa48152f90 0x7faa4815fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41916/0 (socket says 172.21.15.62:41916) 2024-03-28T17:59:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 -- 172.21.15.62:0/582866174 >> v1:172.21.15.81:6789/0 conn(0x7faa4806ccb0 legacy=0x7faa480cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 -- 172.21.15.62:0/582866174 >> v1:172.21.15.62:6789/0 conn(0x7faa48152f90 legacy=0x7faa4815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.182+0000 7faa46800640 1 -- 172.21.15.62:0/582866174 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480b3cb0 con 0x7faa480cf590 2024-03-28T17:59:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.183+0000 7faa46800640 1 --2- 172.21.15.62:0/582866174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa480cf590 0x7faa480b44b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa40009e50 tx=0x7faa4002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b72bfeca7642c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa44a00640 1 -- 172.21.15.62:0/582866174 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4002cb10 con 0x7faa480cf590 2024-03-28T17:59:42.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa44a00640 1 -- 172.21.15.62:0/582866174 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4002ccb0 con 0x7faa480cf590 2024-03-28T17:59:42.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa44a00640 1 -- 172.21.15.62:0/582866174 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4002ce60 con 0x7faa480cf590 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa4cc00640 1 -- 172.21.15.62:0/582866174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa480cf590 msgr2=0x7faa480b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa4cc00640 1 --2- 172.21.15.62:0/582866174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa480cf590 0x7faa480b44b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa40009e50 tx=0x7faa4002bf70 comp rx=0 tx=0).stop 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa4cc00640 1 -- 172.21.15.62:0/582866174 shutdown_connections 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa4cc00640 1 --2- 172.21.15.62:0/582866174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa480cf590 0x7faa480b44b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.184+0000 7faa4cc00640 1 -- 172.21.15.62:0/582866174 >> 172.21.15.62:0/582866174 conn(0x7faa48066fb0 msgr2=0x7faa480673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.185+0000 7faa4cc00640 1 -- 172.21.15.62:0/582866174 shutdown_connections 2024-03-28T17:59:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.185+0000 7faa4cc00640 1 -- 172.21.15.62:0/582866174 wait complete. 2024-03-28T17:59:42.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.186+0000 7faa4cc00640 1 Processor -- start 2024-03-28T17:59:42.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.186+0000 7faa4cc00640 1 -- start start 2024-03-28T17:59:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 0x7faa4809cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa48152f90 0x7faa481e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4815c750 con 0x7faa4806ccb0 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4815c5d0 con 0x7faa480cf590 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.187+0000 7faa4cc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480b49f0 con 0x7faa48152f90 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46424/0 (socket says 172.21.15.62:46424) 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 -- 172.21.15.62:0/3582484559 learned_addr learned my addr 172.21.15.62:0/3582484559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45400640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 0x7faa4809cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa46800640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa48152f90 0x7faa481e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 -- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa48152f90 msgr2=0x7faa481e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa48152f90 0x7faa481e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 -- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 msgr2=0x7faa4809cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 0x7faa4809cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45e00640 1 -- 172.21.15.62:0/3582484559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4809af80 con 0x7faa4806ccb0 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.188+0000 7faa45400640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 0x7faa4809cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa45e00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c3c0 tx=0x7faa3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380074e0 con 0x7faa4806ccb0 2024-03-28T17:59:42.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38007680 con 0x7faa4806ccb0 2024-03-28T17:59:42.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38004a00 con 0x7faa4806ccb0 2024-03-28T17:59:42.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa481e6540 con 0x7faa4806ccb0 2024-03-28T17:59:42.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.189+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa481e6b00 con 0x7faa4806ccb0 2024-03-28T17:59:42.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.190+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4809b150 con 0x7faa4806ccb0 2024-03-28T17:59:42.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.191+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faa3801a020 con 0x7faa4806ccb0 2024-03-28T17:59:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.192+0000 7faa3e000640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 0x7faa2806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.192+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faa3808a1e0 con 0x7faa4806ccb0 2024-03-28T17:59:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.193+0000 7faa45400640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 0x7faa2806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.194+0000 7faa45400640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 0x7faa2806e090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa480cbe20 tx=0x7faa34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.197+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faa38058ce0 con 0x7faa4806ccb0 2024-03-28T17:59:42.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.416+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa4809aa60 con 0x7faa4806ccb0 2024-03-28T17:59:42.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.417+0000 7faa3e000640 1 -- 172.21.15.62:0/3582484559 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa380585e0 con 0x7faa4806ccb0 2024-03-28T17:59:42.417 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:42.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 msgr2=0x7faa2806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 0x7faa2806e090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa480cbe20 tx=0x7faa34009040 comp rx=0 tx=0).stop 2024-03-28T17:59:42.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 msgr2=0x7faa48148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c3c0 tx=0x7faa3800c880 comp rx=0 tx=0).stop 2024-03-28T17:59:42.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 shutdown_connections 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa2806bbd0 0x7faa2806e090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa48152f90 0x7faa481e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa480cf590 0x7faa4809cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 --2- 172.21.15.62:0/3582484559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4806ccb0 0x7faa48148780 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 >> 172.21.15.62:0/3582484559 conn(0x7faa48066fb0 msgr2=0x7faa4815f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 shutdown_connections 2024-03-28T17:59:42.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.420+0000 7faa4cc00640 1 -- 172.21.15.62:0/3582484559 wait complete. 2024-03-28T17:59:42.439 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:42.440 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:42.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.711+0000 7fce99a00640 1 Processor -- start 2024-03-28T17:59:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.712+0000 7fce99a00640 1 -- start start 2024-03-28T17:59:42.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce94155380 0x7fce9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9406cb10 0x7fce94070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce9415e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce940713d0 con 0x7fce9406cb10 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce94155760 con 0x7fce94155380 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.713+0000 7fce99a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9415b810 con 0x7fce94070f80 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9406cb10 0x7fce94070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9406cb10 0x7fce94070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46432/0 (socket says 172.21.15.62:46432) 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 -- 172.21.15.62:0/391724629 learned_addr learned my addr 172.21.15.62:0/391724629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce93e00640 1 --2- 172.21.15.62:0/391724629 >> v2:172.21.15.81:3300/0 conn(0x7fce94155380 0x7fce9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce99000640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce9415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 -- 172.21.15.62:0/391724629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 msgr2=0x7fce9415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:42.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce9415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 -- 172.21.15.62:0/391724629 >> v2:172.21.15.81:3300/0 conn(0x7fce94155380 msgr2=0x7fce9415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:42.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 --2- 172.21.15.62:0/391724629 >> v2:172.21.15.81:3300/0 conn(0x7fce94155380 0x7fce9415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.714+0000 7fce8b400640 1 -- 172.21.15.62:0/391724629 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9415baf0 con 0x7fce9406cb10 2024-03-28T17:59:42.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.715+0000 7fce8b400640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9406cb10 0x7fce94070a40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce94067c80 tx=0x7fce7c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57d2ec425212bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce93400640 1 -- 172.21.15.62:0/391724629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c037070 con 0x7fce9406cb10 2024-03-28T17:59:42.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce93400640 1 -- 172.21.15.62:0/391724629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c004230 con 0x7fce9406cb10 2024-03-28T17:59:42.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce93400640 1 -- 172.21.15.62:0/391724629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c003610 con 0x7fce9406cb10 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 -- 172.21.15.62:0/391724629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9406cb10 msgr2=0x7fce94070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9406cb10 0x7fce94070a40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce94067c80 tx=0x7fce7c02b860 comp rx=0 tx=0).stop 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 -- 172.21.15.62:0/391724629 shutdown_connections 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce9415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 --2- 172.21.15.62:0/391724629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9406cb10 0x7fce94070a40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 --2- 172.21.15.62:0/391724629 >> v2:172.21.15.81:3300/0 conn(0x7fce94155380 0x7fce9415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 -- 172.21.15.62:0/391724629 >> 172.21.15.62:0/391724629 conn(0x7fce940b1730 msgr2=0x7fce940b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.716+0000 7fce99a00640 1 -- 172.21.15.62:0/391724629 shutdown_connections 2024-03-28T17:59:42.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.717+0000 7fce99a00640 1 -- 172.21.15.62:0/391724629 wait complete. 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.718+0000 7fce99a00640 1 Processor -- start 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.718+0000 7fce99a00640 1 -- start start 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce9406cb10 0x7fce941464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce940aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9406cfc0 con 0x7fce94155380 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9415c0b0 con 0x7fce9406cb10 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.719+0000 7fce99a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9406d140 con 0x7fce94070f80 2024-03-28T17:59:42.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46444/0 (socket says 172.21.15.62:46444) 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 -- 172.21.15.62:0/2061360085 learned_addr learned my addr 172.21.15.62:0/2061360085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce93e00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce9406cb10 0x7fce941464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce8b400640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce940aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 -- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 msgr2=0x7fce940aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce940aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 -- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce9406cb10 msgr2=0x7fce941464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce9406cb10 0x7fce941464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 -- 172.21.15.62:0/2061360085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940aa3e0 con 0x7fce94155380 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce8b400640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce940aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.720+0000 7fce99000640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007e60 tx=0x7fce8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.721+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c014620 con 0x7fce94155380 2024-03-28T17:59:42.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.721+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c019070 con 0x7fce94155380 2024-03-28T17:59:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.721+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c014a20 con 0x7fce94155380 2024-03-28T17:59:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.721+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce940aa6c0 con 0x7fce94155380 2024-03-28T17:59:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.721+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce940aac80 con 0x7fce94155380 2024-03-28T17:59:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.722+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce94073490 con 0x7fce94155380 2024-03-28T17:59:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.723+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fce8c014bc0 con 0x7fce94155380 2024-03-28T17:59:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.724+0000 7fce90c00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 0x7fce7006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.724+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fce8c08b450 con 0x7fce94155380 2024-03-28T17:59:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.725+0000 7fce93e00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 0x7fce7006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.726+0000 7fce93e00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 0x7fce7006e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.729+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fce8c059e20 con 0x7fce94155380 2024-03-28T17:59:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.932+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce940cbe60 con 0x7fce94155380 2024-03-28T17:59:42.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.934+0000 7fce90c00640 1 -- 172.21.15.62:0/2061360085 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7fce8c059720 con 0x7fce94155380 2024-03-28T17:59:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:42.935 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 msgr2=0x7fce7006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 0x7fce7006e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).stop 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 msgr2=0x7fce940acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007e60 tx=0x7fce8c00e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 shutdown_connections 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fce7006bbd0 0x7fce7006e090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce94155380 0x7fce940acaa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fce94070f80 0x7fce940aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 --2- 172.21.15.62:0/2061360085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce9406cb10 0x7fce941464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:42.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 >> 172.21.15.62:0/2061360085 conn(0x7fce940b1730 msgr2=0x7fce9406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:42.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 shutdown_connections 2024-03-28T17:59:42.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:42.937+0000 7fce99a00640 1 -- 172.21.15.62:0/2061360085 wait complete. 2024-03-28T17:59:42.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:42.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:43.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.187+0000 7f5616600640 1 Processor -- start 2024-03-28T17:59:43.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.188+0000 7f5616600640 1 -- start start 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5616600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f561015e2c0 0x7f56100ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5616600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56100b3210 0x7f56100d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5616600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5610161bb0 con 0x7f56100cec50 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5616600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5610161e90 con 0x7f56100b3210 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5616600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5610162170 con 0x7f561015e2c0 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5615c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56100b3210 0x7f56100d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5615c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56100b3210 0x7f56100d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46450/0 (socket says 172.21.15.62:46450) 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f5615c00640 1 -- 172.21.15.62:0/3196288687 learned_addr learned my addr 172.21.15.62:0/3196288687 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.189+0000 7f560fe00640 1 --2- 172.21.15.62:0/3196288687 >> v2:172.21.15.81:3300/0 conn(0x7f561015e2c0 0x7f56100ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.190+0000 7f5615c00640 1 -- 172.21.15.62:0/3196288687 >> v2:172.21.15.81:3300/0 conn(0x7f561015e2c0 msgr2=0x7f56100ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.190+0000 7f5615c00640 1 --2- 172.21.15.62:0/3196288687 >> v2:172.21.15.81:3300/0 conn(0x7f561015e2c0 0x7f56100ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.190+0000 7f5615c00640 1 -- 172.21.15.62:0/3196288687 >> v1:172.21.15.81:6789/0 conn(0x7f56100cec50 legacy=0x7f56100cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.190+0000 7f5615c00640 1 -- 172.21.15.62:0/3196288687 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610162450 con 0x7f56100b3210 2024-03-28T17:59:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.191+0000 7f5615c00640 1 --2- 172.21.15.62:0/3196288687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100b3210 0x7f56100d1fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56100ce1a0 tx=0x7f560802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11b55c2cee2747 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.191+0000 7f560f400640 1 -- 172.21.15.62:0/3196288687 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560802ca80 con 0x7f56100b3210 2024-03-28T17:59:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.191+0000 7f560f400640 1 -- 172.21.15.62:0/3196288687 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560802cc20 con 0x7f56100b3210 2024-03-28T17:59:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.191+0000 7f560f400640 1 -- 172.21.15.62:0/3196288687 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080343f0 con 0x7f56100b3210 2024-03-28T17:59:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 -- 172.21.15.62:0/3196288687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100b3210 msgr2=0x7f56100d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 --2- 172.21.15.62:0/3196288687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100b3210 0x7f56100d1fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56100ce1a0 tx=0x7f560802bee0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 -- 172.21.15.62:0/3196288687 shutdown_connections 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 --2- 172.21.15.62:0/3196288687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100b3210 0x7f56100d1fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 --2- 172.21.15.62:0/3196288687 >> v2:172.21.15.81:3300/0 conn(0x7f561015e2c0 0x7f56100ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 -- 172.21.15.62:0/3196288687 >> 172.21.15.62:0/3196288687 conn(0x7f5610066fb0 msgr2=0x7f56100673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 -- 172.21.15.62:0/3196288687 shutdown_connections 2024-03-28T17:59:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.192+0000 7f5616600640 1 -- 172.21.15.62:0/3196288687 wait complete. 2024-03-28T17:59:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.194+0000 7f5616600640 1 Processor -- start 2024-03-28T17:59:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.194+0000 7f5616600640 1 -- start start 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 0x7f56100b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56100b61e0 0x7f56100cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100d24f0 con 0x7f56100cec50 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610162a10 con 0x7f56100b3210 2024-03-28T17:59:43.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f5616600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56100d2670 con 0x7f56100b61e0 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.195+0000 7f560fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46462/0 (socket says 172.21.15.62:46462) 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 -- 172.21.15.62:0/1412537592 learned_addr learned my addr 172.21.15.62:0/1412537592 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f5615200640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 0x7f56100b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 -- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56100b61e0 msgr2=0x7f56100cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f5615c00640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56100b61e0 0x7f56100cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56100b61e0 0x7f56100cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 -- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 msgr2=0x7f56100b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 0x7f56100b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 -- 172.21.15.62:0/1412537592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56100b39d0 con 0x7f56100cec50 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f5615200640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 0x7f56100b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:43.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.196+0000 7f560fe00640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d5f0 tx=0x7f55fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.197+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0125f0 con 0x7f56100cec50 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.197+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc019070 con 0x7f56100cec50 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.197+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0129d0 con 0x7f56100cec50 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.197+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56100b3cb0 con 0x7f56100cec50 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.197+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5610155a60 con 0x7f56100cec50 2024-03-28T17:59:43.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.198+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561004f400 con 0x7f56100cec50 2024-03-28T17:59:43.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.199+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f55fc012b70 con 0x7f56100cec50 2024-03-28T17:59:43.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.200+0000 7f560cc00640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 0x7f55f006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.200+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f55fc08afc0 con 0x7f56100cec50 2024-03-28T17:59:43.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.200+0000 7f5615200640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 0x7f55f006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.202+0000 7f5615200640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 0x7f55f006e090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5600003eb0 tx=0x7f5600009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.204+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f55fc059990 con 0x7f56100cec50 2024-03-28T17:59:43.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.411+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56100c72c0 con 0x7f56100cec50 2024-03-28T17:59:43.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.412+0000 7f560cc00640 1 -- 172.21.15.62:0/1412537592 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55fc059290 con 0x7f56100cec50 2024-03-28T17:59:43.412 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:43.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 msgr2=0x7f55f006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 0x7f55f006e090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5600003eb0 tx=0x7f5600009040 comp rx=0 tx=0).stop 2024-03-28T17:59:43.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 msgr2=0x7f56100cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d5f0 tx=0x7f55fc00dab0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 shutdown_connections 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f55f006bbd0 0x7f55f006e090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56100b61e0 0x7f56100cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56100cec50 0x7f56100cba00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 --2- 172.21.15.62:0/1412537592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56100b3210 0x7f56100b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 >> 172.21.15.62:0/1412537592 conn(0x7f5610066fb0 msgr2=0x7f56100b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 shutdown_connections 2024-03-28T17:59:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.415+0000 7f5616600640 1 -- 172.21.15.62:0/1412537592 wait complete. 2024-03-28T17:59:43.436 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:43.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.664+0000 7f0119400640 1 Processor -- start 2024-03-28T17:59:43.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.664+0000 7f0119400640 1 -- start start 2024-03-28T17:59:43.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0119400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0114073f90 0x7f01140af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0119400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01140cfde0 con 0x7f011415c290 2024-03-28T17:59:43.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0119400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0114072f80 con 0x7f01140d0090 2024-03-28T17:59:43.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0119400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01140d1050 con 0x7f0114073f90 2024-03-28T17:59:43.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0113400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f011415c290 0x7f01140d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41924/0 (socket says 172.21.15.62:41924) 2024-03-28T17:59:43.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.665+0000 7f0113400640 1 -- 172.21.15.62:0/2019362844 learned_addr learned my addr 172.21.15.62:0/2019362844 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:43.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0118a00640 1 --2- 172.21.15.62:0/2019362844 >> v2:172.21.15.81:3300/0 conn(0x7f0114073f90 0x7f01140af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006813041 0 0) 0x7f01140cfde0 con 0x7f011415c290 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f4003520 con 0x7f011415c290 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406157971 0 0) 0x7f00f4003520 con 0x7f011415c290 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 >> v2:172.21.15.81:3300/0 conn(0x7f0114073f90 msgr2=0x7f01140af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 --2- 172.21.15.62:0/2019362844 >> v2:172.21.15.81:3300/0 conn(0x7f0114073f90 0x7f01140af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 >> v1:172.21.15.81:6789/0 conn(0x7f01140d0090 legacy=0x7f01140cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114073260 con 0x7f011415c290 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f00fc0019f0 con 0x7f011415c290 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.666+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 >> v1:172.21.15.62:6789/0 conn(0x7f011415c290 legacy=0x7f01140d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 0x7f00f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4004800 0x7f00f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00f4005210 0x7f00f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140d1050 con 0x7f00f4003a00 2024-03-28T17:59:43.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400c160 con 0x7f00f4005210 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400c360 con 0x7f00f4004800 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 0x7f00f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 -- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4004800 msgr2=0x7f00f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4004800 0x7f00f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 -- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00f4005210 msgr2=0x7f00f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00f4005210 0x7f00f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0118a00640 1 -- 172.21.15.62:0/2019362844 reap_dead start 2024-03-28T17:59:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.667+0000 7f0113e00640 1 -- 172.21.15.62:0/2019362844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4003520 con 0x7f00f4003a00 2024-03-28T17:59:43.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.668+0000 7f0113e00640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 0x7f00f4003e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0104001720 tx=0x7f00f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522cc8632b7df081 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104001370 con 0x7f00f4003a00 2024-03-28T17:59:43.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01040052d0 con 0x7f00f4003a00 2024-03-28T17:59:43.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0112a00640 1 -- 172.21.15.62:0/2019362844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104003830 con 0x7f00f4003a00 2024-03-28T17:59:43.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 -- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 msgr2=0x7f00f4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 0x7f00f4003e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0104001720 tx=0x7f00f4005830 comp rx=0 tx=0).stop 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 -- 172.21.15.62:0/2019362844 shutdown_connections 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 --2- 172.21.15.62:0/2019362844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00f4003a00 0x7f00f4003e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 -- 172.21.15.62:0/2019362844 >> 172.21.15.62:0/2019362844 conn(0x7f01140b5790 msgr2=0x7f0114072530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 -- 172.21.15.62:0/2019362844 shutdown_connections 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.669+0000 7f0119400640 1 -- 172.21.15.62:0/2019362844 wait complete. 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.671+0000 7f0119400640 1 Processor -- start 2024-03-28T17:59:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.671+0000 7f0119400640 1 -- start start 2024-03-28T17:59:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4003a00 0x7f011415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 0x7f01141f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140af7d0 con 0x7f0114150a60 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140af650 con 0x7f011415af70 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0119400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01140cef60 con 0x7f00f4003a00 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0113400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0113400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46480/0 (socket says 172.21.15.62:46480) 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.672+0000 7f0113400640 1 -- 172.21.15.62:0/2731427223 learned_addr learned my addr 172.21.15.62:0/2731427223 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0118a00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 0x7f01141f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113e00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4003a00 0x7f011415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 -- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4003a00 msgr2=0x7f011415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4003a00 0x7f011415edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 -- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 msgr2=0x7f01141f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 0x7f01141f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 -- 172.21.15.62:0/2731427223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011415e300 con 0x7f0114150a60 2024-03-28T17:59:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0118a00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 0x7f01141f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0113400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00ca70 tx=0x7f00fc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc018070 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc00a6a0 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01141f5090 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.673+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00aa90 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.674+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011415e8a0 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.675+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011415eaa0 con 0x7f0114150a60 2024-03-28T17:59:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.676+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f00fc01b070 con 0x7f0114150a60 2024-03-28T17:59:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.677+0000 7f010be00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 0x7f00f006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.677+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f00fc08a370 con 0x7f0114150a60 2024-03-28T17:59:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.678+0000 7f0113e00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 0x7f00f006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:43.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.679+0000 7f0113e00640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 0x7f00f006e090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0104033000 tx=0x7f0104009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:43.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.682+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f011415eaa0 con 0x7f0114150a60 2024-03-28T17:59:43.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.890+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f011404f400 con 0x7f0114150a60 2024-03-28T17:59:43.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.890+0000 7f010be00640 1 -- 172.21.15.62:0/2731427223 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00fc058d50 con 0x7f0114150a60 2024-03-28T17:59:43.891 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 msgr2=0x7f00f006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 0x7f00f006e090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0104033000 tx=0x7f0104009030 comp rx=0 tx=0).stop 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 msgr2=0x7f011415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00ca70 tx=0x7f00fc00cf30 comp rx=0 tx=0).stop 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 shutdown_connections 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f00f006bbd0 0x7f00f006e090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f011415af70 0x7f01141f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0114150a60 0x7f011415aa30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 --2- 172.21.15.62:0/2731427223 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f00f4003a00 0x7f011415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:43.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.893+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 >> 172.21.15.62:0/2731427223 conn(0x7f01140b5790 msgr2=0x7f01140b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:43.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.894+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 shutdown_connections 2024-03-28T17:59:43.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:43.894+0000 7f0119400640 1 -- 172.21.15.62:0/2731427223 wait complete. 2024-03-28T17:59:43.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:44.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:59:44.138 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T17:59:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.143+0000 7faed3e00640 1 Processor -- start 2024-03-28T17:59:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.143+0000 7faed3e00640 1 -- start start 2024-03-28T17:59:44.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.145+0000 7faed3e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faed4073f90 0x7faed40af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.145+0000 7faed3e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed40cfde0 con 0x7faed40d0090 2024-03-28T17:59:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.145+0000 7faed3e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed4072f80 con 0x7faed415c290 2024-03-28T17:59:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.145+0000 7faed3e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed40d1050 con 0x7faed4073f90 2024-03-28T17:59:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.146+0000 7faed2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faed40d0090 0x7faed40cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45314/0 (socket says 172.21.15.62:45314) 2024-03-28T17:59:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.146+0000 7faed2a00640 1 -- 172.21.15.62:0/1831706128 learned_addr learned my addr 172.21.15.62:0/1831706128 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.146+0000 7faed3400640 1 --2- 172.21.15.62:0/1831706128 >> v2:172.21.15.81:3300/0 conn(0x7faed4073f90 0x7faed40af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.146+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919054115 0 0) 0x7faed40cfde0 con 0x7faed40d0090 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.146+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb8003520 con 0x7faed40d0090 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983180692 0 0) 0x7faeb8003520 con 0x7faed40d0090 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 >> v2:172.21.15.81:3300/0 conn(0x7faed4073f90 msgr2=0x7faed40af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 --2- 172.21.15.62:0/1831706128 >> v2:172.21.15.81:3300/0 conn(0x7faed4073f90 0x7faed40af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 >> v1:172.21.15.81:6789/0 conn(0x7faed415c290 legacy=0x7faed40d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed4073260 con 0x7faed40d0090 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7faec4001670 con 0x7faed40d0090 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 >> v1:172.21.15.62:6789/0 conn(0x7faed40d0090 legacy=0x7faed40cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8003a00 0x7faeb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 0x7faeb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faeb8005210 0x7faeb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40d1050 con 0x7faeb8004800 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb800c160 con 0x7faeb8003a00 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb800c360 con 0x7faeb8005210 2024-03-28T17:59:44.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.147+0000 7faecbe00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 0x7faeb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faed2a00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8003a00 0x7faeb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faecbe00640 1 -- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faeb8005210 msgr2=0x7faeb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faecbe00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faeb8005210 0x7faeb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faecbe00640 1 -- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8003a00 msgr2=0x7faeb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faecbe00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8003a00 0x7faeb8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faecbe00640 1 -- 172.21.15.62:0/1831706128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8003520 con 0x7faeb8004800 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.148+0000 7faed3400640 1 -- 172.21.15.62:0/1831706128 reap_dead start 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.149+0000 7faecbe00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 0x7faeb8004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faed409dd50 tx=0x7faeb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e46a675e7c21d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.149+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc001370 con 0x7faeb8004800 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.149+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc0052d0 con 0x7faeb8004800 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.149+0000 7faed2000640 1 -- 172.21.15.62:0/1831706128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc003830 con 0x7faeb8004800 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 -- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 msgr2=0x7faeb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 0x7faeb8004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faed409dd50 tx=0x7faeb8005830 comp rx=0 tx=0).stop 2024-03-28T17:59:44.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 -- 172.21.15.62:0/1831706128 shutdown_connections 2024-03-28T17:59:44.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 --2- 172.21.15.62:0/1831706128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8004800 0x7faeb8004c80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 -- 172.21.15.62:0/1831706128 >> 172.21.15.62:0/1831706128 conn(0x7faed40b5790 msgr2=0x7faed4072530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:44.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 -- 172.21.15.62:0/1831706128 shutdown_connections 2024-03-28T17:59:44.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.150+0000 7faed3e00640 1 -- 172.21.15.62:0/1831706128 wait complete. 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.152+0000 7faed3e00640 1 Processor -- start 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.152+0000 7faed3e00640 1 -- start start 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8004800 0x7faed414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 0x7faed415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40af7d0 con 0x7faed41f1fd0 2024-03-28T17:59:44.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40af650 con 0x7faeb8004800 2024-03-28T17:59:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40cef60 con 0x7faed41f1b50 2024-03-28T17:59:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32790/0 (socket says 172.21.15.62:32790) 2024-03-28T17:59:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.153+0000 7faed3400640 1 -- 172.21.15.62:0/1460239370 learned_addr learned my addr 172.21.15.62:0/1460239370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faecbe00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 0x7faed415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed2a00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8004800 0x7faed414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 -- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 msgr2=0x7faed415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 0x7faed415ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 -- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8004800 msgr2=0x7faed414cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8004800 0x7faed414cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 -- 172.21.15.62:0/1460239370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed414cff0 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faecbe00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 0x7faed415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.154+0000 7faed3400640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faed4072200 tx=0x7faecc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.155+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ba80 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.155+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc01a070 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.155+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00be40 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.155+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed409aeb0 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.155+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed409b470 con 0x7faed41f1fd0 2024-03-28T17:59:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.156+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed404f400 con 0x7faed41f1fd0 2024-03-28T17:59:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.157+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faecc013070 con 0x7faed41f1fd0 2024-03-28T17:59:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.158+0000 7faecb400640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 0x7faeb006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.158+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faecc08abf0 con 0x7faed41f1fd0 2024-03-28T17:59:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.159+0000 7faed2a00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 0x7faeb006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.160+0000 7faed2a00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 0x7faeb006e090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faec4001050 tx=0x7faec4007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.163+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faecc0596f0 con 0x7faed41f1fd0 2024-03-28T17:59:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.368+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faed40ae210 con 0x7faed41f1fd0 2024-03-28T17:59:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.369+0000 7faecb400640 1 -- 172.21.15.62:0/1460239370 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7faecc058ff0 con 0x7faed41f1fd0 2024-03-28T17:59:44.369 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.371+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 msgr2=0x7faeb006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.371+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 0x7faeb006e090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faec4001050 tx=0x7faec4007040 comp rx=0 tx=0).stop 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.371+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 msgr2=0x7faed40d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.371+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faed4072200 tx=0x7faecc00b480 comp rx=0 tx=0).stop 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 shutdown_connections 2024-03-28T17:59:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faeb006bbd0 0x7faeb006e090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faed41f1fd0 0x7faed40d0470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faed41f1b50 0x7faed415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 --2- 172.21.15.62:0/1460239370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb8004800 0x7faed414cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 >> 172.21.15.62:0/1460239370 conn(0x7faed40b5790 msgr2=0x7faed4158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 shutdown_connections 2024-03-28T17:59:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.372+0000 7faed3e00640 1 -- 172.21.15.62:0/1460239370 wait complete. 2024-03-28T17:59:44.392 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:44.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.594+0000 7fd731a00640 1 Processor -- start 2024-03-28T17:59:44.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.595+0000 7fd731a00640 1 -- start start 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd72c15dd60 0x7fd72c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd72c0b99f0 0x7fd72c0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c0b6c30 con 0x7fd72c15dd60 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c0b6080 con 0x7fd72c0b99f0 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd731a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b6e60 con 0x7fd72c0b5c30 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.596+0000 7fd72be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd72c15dd60 0x7fd72c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd72c15dd60 0x7fd72c0b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32798/0 (socket says 172.21.15.62:32798) 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 -- 172.21.15.62:0/2878737654 learned_addr learned my addr 172.21.15.62:0/2878737654 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:44.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd731000640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72b400640 1 --2- 172.21.15.62:0/2878737654 >> v2:172.21.15.81:3300/0 conn(0x7fd72c0b99f0 0x7fd72c0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 -- 172.21.15.62:0/2878737654 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 msgr2=0x7fd72c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 -- 172.21.15.62:0/2878737654 >> v2:172.21.15.81:3300/0 conn(0x7fd72c0b99f0 msgr2=0x7fd72c0b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 --2- 172.21.15.62:0/2878737654 >> v2:172.21.15.81:3300/0 conn(0x7fd72c0b99f0 0x7fd72c0b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.597+0000 7fd72be00640 1 -- 172.21.15.62:0/2878737654 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0b7100 con 0x7fd72c15dd60 2024-03-28T17:59:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.598+0000 7fd72be00640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0b94b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd72c074f20 tx=0x7fd71c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415a283f06a928d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd72aa00640 1 -- 172.21.15.62:0/2878737654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c034070 con 0x7fd72c15dd60 2024-03-28T17:59:44.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd72aa00640 1 -- 172.21.15.62:0/2878737654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c030860 con 0x7fd72c15dd60 2024-03-28T17:59:44.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd72aa00640 1 -- 172.21.15.62:0/2878737654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c030c30 con 0x7fd72c15dd60 2024-03-28T17:59:44.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 -- 172.21.15.62:0/2878737654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 msgr2=0x7fd72c0b94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0b94b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd72c074f20 tx=0x7fd71c02b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 -- 172.21.15.62:0/2878737654 shutdown_connections 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 --2- 172.21.15.62:0/2878737654 >> v2:172.21.15.81:3300/0 conn(0x7fd72c0b99f0 0x7fd72c0b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 --2- 172.21.15.62:0/2878737654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0b94b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 -- 172.21.15.62:0/2878737654 >> 172.21.15.62:0/2878737654 conn(0x7fd72c071ae0 msgr2=0x7fd72c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.599+0000 7fd731a00640 1 -- 172.21.15.62:0/2878737654 shutdown_connections 2024-03-28T17:59:44.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.600+0000 7fd731a00640 1 -- 172.21.15.62:0/2878737654 wait complete. 2024-03-28T17:59:44.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.601+0000 7fd731a00640 1 Processor -- start 2024-03-28T17:59:44.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.601+0000 7fd731a00640 1 -- start start 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd72c0b99f0 0x7fd72c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b79a0 con 0x7fd72c15dd60 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b76c0 con 0x7fd72c0b99f0 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.602+0000 7fd731a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c15e140 con 0x7fd72c0b5c30 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32808/0 (socket says 172.21.15.62:32808) 2024-03-28T17:59:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 -- 172.21.15.62:0/3926237231 learned_addr learned my addr 172.21.15.62:0/3926237231 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd72b400640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd72c0b99f0 0x7fd72c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd72be00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 -- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 msgr2=0x7fd72c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 -- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd72c0b99f0 msgr2=0x7fd72c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd72c0b99f0 0x7fd72c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.603+0000 7fd731000640 1 -- 172.21.15.62:0/3926237231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c1603d0 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd731000640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d800 tx=0x7fd72400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724018020 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7240096c0 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724009a90 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c160990 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.604+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c160f50 con 0x7fd72c15dd60 2024-03-28T17:59:44.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.605+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c072b90 con 0x7fd72c15dd60 2024-03-28T17:59:44.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.606+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd72401b020 con 0x7fd72c15dd60 2024-03-28T17:59:44.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.607+0000 7fd723e00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 0x7fd70806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:44.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.607+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd72408ae10 con 0x7fd72c15dd60 2024-03-28T17:59:44.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.608+0000 7fd72be00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 0x7fd70806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:44.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.609+0000 7fd72be00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 0x7fd70806e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd71c036000 tx=0x7fd71c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:44.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.612+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd72408e030 con 0x7fd72c15dd60 2024-03-28T17:59:44.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.825+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd72c0cfd60 con 0x7fd72c15dd60 2024-03-28T17:59:44.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.826+0000 7fd723e00640 1 -- 172.21.15.62:0/3926237231 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd724022030 con 0x7fd72c15dd60 2024-03-28T17:59:44.827 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 msgr2=0x7fd70806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 0x7fd70806e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd71c036000 tx=0x7fd71c03b040 comp rx=0 tx=0).stop 2024-03-28T17:59:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 msgr2=0x7fd72c0d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:44.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d800 tx=0x7fd72400dcc0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 shutdown_connections 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd70806bbd0 0x7fd70806e090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd72c15dd60 0x7fd72c0d3220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd72c0b99f0 0x7fd72c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 --2- 172.21.15.62:0/3926237231 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd72c0b5c30 0x7fd72c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 >> 172.21.15.62:0/3926237231 conn(0x7fd72c071ae0 msgr2=0x7fd72c0b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 shutdown_connections 2024-03-28T17:59:44.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:44.829+0000 7fd731a00640 1 -- 172.21.15.62:0/3926237231 wait complete. 2024-03-28T17:59:44.849 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi062.front.sepia.ceph.com... 2024-03-28T17:59:44.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:44.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-28T17:59:44.946 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T17:59:44.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:59:44.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:44.947 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T17:59:45.019 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:45.019 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:45.072 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:45.072 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:45.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.390+0000 7f8966000640 1 Processor -- start 2024-03-28T17:59:45.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.390+0000 7f8966000640 1 -- start start 2024-03-28T17:59:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f8966000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 0x7f896015f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f8966000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8960072ab0 0x7f896015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f8966000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600cebb0 con 0x7f896006cd80 2024-03-28T17:59:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f8966000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89600cee90 con 0x7f8960072ab0 2024-03-28T17:59:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f8966000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600cf170 con 0x7f89601456d0 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f895fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8960072ab0 0x7f896015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f895fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8960072ab0 0x7f896015fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32820/0 (socket says 172.21.15.62:32820) 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.392+0000 7f895fe00640 1 -- 172.21.15.62:0/493631900 learned_addr learned my addr 172.21.15.62:0/493631900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f8964c00640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 0x7f896015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f895fe00640 1 -- 172.21.15.62:0/493631900 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 msgr2=0x7f896015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f895fe00640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 0x7f896015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f895fe00640 1 -- 172.21.15.62:0/493631900 >> v1:172.21.15.81:6789/0 conn(0x7f896006cd80 legacy=0x7f896006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f895f400640 1 -- 172.21.15.62:0/493631900 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522910608 0 0) 0x7f89600cebb0 con 0x7f896006cd80 2024-03-28T17:59:45.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f895fe00640 1 -- 172.21.15.62:0/493631900 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600cf450 con 0x7f8960072ab0 2024-03-28T17:59:45.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.393+0000 7f8964c00640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 0x7f896015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:45.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.394+0000 7f895fe00640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8960072ab0 0x7f896015fec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f894c009850 tx=0x7f894c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94784ecbfb1b8ece server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.394+0000 7f895f400640 1 -- 172.21.15.62:0/493631900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c004230 con 0x7f8960072ab0 2024-03-28T17:59:45.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.394+0000 7f895f400640 1 -- 172.21.15.62:0/493631900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c0043d0 con 0x7f8960072ab0 2024-03-28T17:59:45.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.394+0000 7f895f400640 1 -- 172.21.15.62:0/493631900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c031830 con 0x7f8960072ab0 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 -- 172.21.15.62:0/493631900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8960072ab0 msgr2=0x7f896015fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8960072ab0 0x7f896015fec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f894c009850 tx=0x7f894c02b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 -- 172.21.15.62:0/493631900 shutdown_connections 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8960072ab0 0x7f896015fec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 --2- 172.21.15.62:0/493631900 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f89601456d0 0x7f896015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 -- 172.21.15.62:0/493631900 >> 172.21.15.62:0/493631900 conn(0x7f89600b1730 msgr2=0x7f89600b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 -- 172.21.15.62:0/493631900 shutdown_connections 2024-03-28T17:59:45.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.395+0000 7f8966000640 1 -- 172.21.15.62:0/493631900 wait complete. 2024-03-28T17:59:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.397+0000 7f8966000640 1 Processor -- start 2024-03-28T17:59:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.397+0000 7f8966000640 1 -- start start 2024-03-28T17:59:45.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 0x7f896013b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8960072ab0 0x7f89600b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960071a50 con 0x7f89601456d0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600cf730 con 0x7f896006cd80 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.398+0000 7f8966000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8960071bd0 con 0x7f8960072ab0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32830/0 (socket says 172.21.15.62:32830) 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 -- 172.21.15.62:0/3727056860 learned_addr learned my addr 172.21.15.62:0/3727056860 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f895fe00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8960072ab0 0x7f89600b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8964c00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 0x7f896013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 -- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8960072ab0 msgr2=0x7f89600b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8960072ab0 0x7f89600b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 -- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 msgr2=0x7f896013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 0x7f896013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8965600640 1 -- 172.21.15.62:0/3727056860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600b2b30 con 0x7f89601456d0 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.399+0000 7f8964c00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 0x7f896013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:45.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f8965600640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f895800c760 tx=0x7f895800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958003da0 con 0x7f89601456d0 2024-03-28T17:59:45.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958005070 con 0x7f89601456d0 2024-03-28T17:59:45.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958005460 con 0x7f89601456d0 2024-03-28T17:59:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600b2e10 con 0x7f89601456d0 2024-03-28T17:59:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.400+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89600b33d0 con 0x7f89601456d0 2024-03-28T17:59:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.401+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8960073490 con 0x7f89601456d0 2024-03-28T17:59:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.402+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8958005600 con 0x7f89601456d0 2024-03-28T17:59:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.403+0000 7f895cc00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 0x7f894006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.403+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f895808a960 con 0x7f89601456d0 2024-03-28T17:59:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.404+0000 7f8964c00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 0x7f894006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.405+0000 7f8964c00640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 0x7f894006e090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8954006fd0 tx=0x7f8954008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.408+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f8958059440 con 0x7f89601456d0 2024-03-28T17:59:45.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.611+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8960141760 con 0x7f89601456d0 2024-03-28T17:59:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.613+0000 7f895cc00640 1 -- 172.21.15.62:0/3727056860 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f8958058d40 con 0x7f89601456d0 2024-03-28T17:59:45.614 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:45.614 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 msgr2=0x7f894006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 0x7f894006e090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8954006fd0 tx=0x7f8954008040 comp rx=0 tx=0).stop 2024-03-28T17:59:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 msgr2=0x7f896013d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f895800c760 tx=0x7f895800cc20 comp rx=0 tx=0).stop 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 shutdown_connections 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f894006bbd0 0x7f894006e090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89601456d0 0x7f896013d070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8960072ab0 0x7f89600b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 --2- 172.21.15.62:0/3727056860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896006cd80 0x7f896013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 >> 172.21.15.62:0/3727056860 conn(0x7f89600b1730 msgr2=0x7f89600ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 shutdown_connections 2024-03-28T17:59:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.616+0000 7f8966000640 1 -- 172.21.15.62:0/3727056860 wait complete. 2024-03-28T17:59:45.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:45.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.853+0000 7ff03b000640 1 Processor -- start 2024-03-28T17:59:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.853+0000 7ff03b000640 1 -- start start 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.854+0000 7ff03b000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0340b15c0 0x7ff034160190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff03b000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340b44f0 0x7ff0340b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff03b000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03415bec0 con 0x7ff0340aee00 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff03b000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0340b4e70 con 0x7ff0340b15c0 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff03b000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340b1bf0 con 0x7ff0340b44f0 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff0340aee00 0x7ff03415ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45330/0 (socket says 172.21.15.62:45330) 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039c00640 1 -- 172.21.15.62:0/16012408 learned_addr learned my addr 172.21.15.62:0/16012408 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039200640 1 --2- 172.21.15.62:0/16012408 >> v2:172.21.15.62:3300/0 conn(0x7ff0340b15c0 0x7ff034160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff03a600640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340b44f0 0x7ff0340b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff033e00640 1 -- 172.21.15.62:0/16012408 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417786813 0 0) 0x7ff03415bec0 con 0x7ff0340aee00 2024-03-28T17:59:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039200640 1 -- 172.21.15.62:0/16012408 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340b44f0 msgr2=0x7ff0340b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039200640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340b44f0 0x7ff0340b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039200640 1 -- 172.21.15.62:0/16012408 >> v1:172.21.15.62:6789/0 conn(0x7ff0340aee00 legacy=0x7ff03415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.855+0000 7ff039200640 1 -- 172.21.15.62:0/16012408 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340b1ed0 con 0x7ff0340b15c0 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.857+0000 7ff039200640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b15c0 0x7ff034160190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0200097a0 tx=0x7ff02002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e8ef0507a6052a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.857+0000 7ff033e00640 1 -- 172.21.15.62:0/16012408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020035020 con 0x7ff0340b15c0 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.857+0000 7ff033e00640 1 -- 172.21.15.62:0/16012408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff020031830 con 0x7ff0340b15c0 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.857+0000 7ff033e00640 1 -- 172.21.15.62:0/16012408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020031c00 con 0x7ff0340b15c0 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 -- 172.21.15.62:0/16012408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b15c0 msgr2=0x7ff034160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b15c0 0x7ff034160190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0200097a0 tx=0x7ff02002b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 -- 172.21.15.62:0/16012408 shutdown_connections 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340b44f0 0x7ff0340b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 --2- 172.21.15.62:0/16012408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b15c0 0x7ff034160190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 -- 172.21.15.62:0/16012408 >> 172.21.15.62:0/16012408 conn(0x7ff034066dd0 msgr2=0x7ff0340671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:45.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 -- 172.21.15.62:0/16012408 shutdown_connections 2024-03-28T17:59:45.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.858+0000 7ff03b000640 1 -- 172.21.15.62:0/16012408 wait complete. 2024-03-28T17:59:45.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.860+0000 7ff03b000640 1 Processor -- start 2024-03-28T17:59:45.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.860+0000 7ff03b000640 1 -- start start 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 0x7ff03414f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340b15c0 0x7ff0340cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03415fc70 con 0x7ff0340b44f0 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03415f990 con 0x7ff0340b15c0 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03b000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340af1e0 con 0x7ff0340aee00 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32850/0 (socket says 172.21.15.62:32850) 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff03a600640 1 -- 172.21.15.62:0/3110635291 learned_addr learned my addr 172.21.15.62:0/3110635291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff039c00640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 0x7ff03414f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.861+0000 7ff039200640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340b15c0 0x7ff0340cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 -- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 msgr2=0x7ff03414f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 0x7ff03414f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 -- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340b15c0 msgr2=0x7ff0340cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340b15c0 0x7ff0340cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 -- 172.21.15.62:0/3110635291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03415e340 con 0x7ff0340b44f0 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff039c00640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 0x7ff03414f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03a600640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff02c007e90 tx=0x7ff02c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c014420 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c019070 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c014800 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034158b20 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.862+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034159360 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.863+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0340d0970 con 0x7ff0340b44f0 2024-03-28T17:59:45.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.864+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff02c0149a0 con 0x7ff0340b44f0 2024-03-28T17:59:45.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.865+0000 7ff031600640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 0x7ff01406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.865+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff02c08b450 con 0x7ff0340b44f0 2024-03-28T17:59:45.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.866+0000 7ff039c00640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 0x7ff01406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.867+0000 7ff039c00640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 0x7ff01406e0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff028000f60 tx=0x7ff028007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:45.870+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff0340d0970 con 0x7ff0340b44f0 2024-03-28T17:59:46.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.072+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff03415e510 con 0x7ff0340b44f0 2024-03-28T17:59:46.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.073+0000 7ff031600640 1 -- 172.21.15.62:0/3110635291 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff034158b20 con 0x7ff0340b44f0 2024-03-28T17:59:46.073 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 msgr2=0x7ff01406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 0x7ff01406e0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff028000f60 tx=0x7ff028007c10 comp rx=0 tx=0).stop 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 msgr2=0x7ff034155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff02c007e90 tx=0x7ff02c00e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 shutdown_connections 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff01406bc00 0x7ff01406e0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0340b44f0 0x7ff034155560 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340b15c0 0x7ff0340cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 --2- 172.21.15.62:0/3110635291 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0340aee00 0x7ff03414f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 >> 172.21.15.62:0/3110635291 conn(0x7ff034066dd0 msgr2=0x7ff03415c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:46.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 shutdown_connections 2024-03-28T17:59:46.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.076+0000 7ff03b000640 1 -- 172.21.15.62:0/3110635291 wait complete. 2024-03-28T17:59:46.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:46.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.314+0000 7fa09b200640 1 Processor -- start 2024-03-28T17:59:46.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.315+0000 7fa09b200640 1 -- start start 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa09b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0940b15c0 0x7fa094160190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa09b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09415bec0 con 0x7fa0940aee00 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa09b200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940b1bf0 con 0x7fa0940b44f0 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa09b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940b1ed0 con 0x7fa0940b15c0 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa099e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa0940aee00 0x7fa09415ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45332/0 (socket says 172.21.15.62:45332) 2024-03-28T17:59:46.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa099e00640 1 -- 172.21.15.62:0/990390465 learned_addr learned my addr 172.21.15.62:0/990390465 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.316+0000 7fa099400640 1 --2- 172.21.15.62:0/990390465 >> v2:172.21.15.62:3300/0 conn(0x7fa0940b15c0 0x7fa094160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.317+0000 7fa098a00640 1 -- 172.21.15.62:0/990390465 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570153641 0 0) 0x7fa09415bec0 con 0x7fa0940aee00 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.317+0000 7fa098a00640 1 -- 172.21.15.62:0/990390465 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa07c003520 con 0x7fa0940aee00 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.317+0000 7fa099400640 1 -- 172.21.15.62:0/990390465 >> v1:172.21.15.81:6789/0 conn(0x7fa0940b44f0 legacy=0x7fa0940b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.317+0000 7fa099400640 1 -- 172.21.15.62:0/990390465 >> v1:172.21.15.62:6789/0 conn(0x7fa0940aee00 legacy=0x7fa09415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.317+0000 7fa099400640 1 -- 172.21.15.62:0/990390465 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09415f6b0 con 0x7fa0940b15c0 2024-03-28T17:59:46.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.318+0000 7fa099400640 1 --2- 172.21.15.62:0/990390465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa094160190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0840097a0 tx=0x7fa08402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb23f9bd2135487e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.318+0000 7fa098a00640 1 -- 172.21.15.62:0/990390465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084035020 con 0x7fa0940b15c0 2024-03-28T17:59:46.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.318+0000 7fa098a00640 1 -- 172.21.15.62:0/990390465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084031830 con 0x7fa0940b15c0 2024-03-28T17:59:46.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.318+0000 7fa098a00640 1 -- 172.21.15.62:0/990390465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084031be0 con 0x7fa0940b15c0 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 -- 172.21.15.62:0/990390465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 msgr2=0x7fa094160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 --2- 172.21.15.62:0/990390465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa094160190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0840097a0 tx=0x7fa08402b8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 -- 172.21.15.62:0/990390465 shutdown_connections 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 --2- 172.21.15.62:0/990390465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa094160190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 -- 172.21.15.62:0/990390465 >> 172.21.15.62:0/990390465 conn(0x7fa094066dd0 msgr2=0x7fa0940671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 -- 172.21.15.62:0/990390465 shutdown_connections 2024-03-28T17:59:46.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.319+0000 7fa09b200640 1 -- 172.21.15.62:0/990390465 wait complete. 2024-03-28T17:59:46.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.321+0000 7fa09b200640 1 Processor -- start 2024-03-28T17:59:46.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.321+0000 7fa09b200640 1 -- start start 2024-03-28T17:59:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0940aee00 0x7fa0941500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0940b44f0 0x7fa0941f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940af1e0 con 0x7fa0940b15c0 2024-03-28T17:59:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09415fc70 con 0x7fa0940aee00 2024-03-28T17:59:46.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa09b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940b4930 con 0x7fa0940b44f0 2024-03-28T17:59:46.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa099400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa099400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32864/0 (socket says 172.21.15.62:32864) 2024-03-28T17:59:46.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.322+0000 7fa099400640 1 -- 172.21.15.62:0/1919857113 learned_addr learned my addr 172.21.15.62:0/1919857113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099e00640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0940aee00 0x7fa0941500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa09a800640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0940b44f0 0x7fa0941f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 -- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0940b44f0 msgr2=0x7fa0941f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0940b44f0 0x7fa0941f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 -- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0940aee00 msgr2=0x7fa0941500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0940aee00 0x7fa0941500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 -- 172.21.15.62:0/1919857113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0941f2850 con 0x7fa0940b15c0 2024-03-28T17:59:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.323+0000 7fa099400640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa08402b910 tx=0x7fa0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.324+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084041070 con 0x7fa0940b15c0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.324+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0941565c0 con 0x7fa0940b15c0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.324+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08403b840 con 0x7fa0940b15c0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.324+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084035040 con 0x7fa0940b15c0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.324+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa094156290 con 0x7fa0940b15c0 2024-03-28T17:59:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.325+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09404f400 con 0x7fa0940b15c0 2024-03-28T17:59:46.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.326+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa08404c060 con 0x7fa0940b15c0 2024-03-28T17:59:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.327+0000 7fa08e000640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 0x7fa07806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.327+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa0840acbd0 con 0x7fa0940b15c0 2024-03-28T17:59:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.328+0000 7fa099e00640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 0x7fa07806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.329+0000 7fa099e00640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 0x7fa07806e090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa088007fd0 tx=0x7fa088009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.331+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa084031800 con 0x7fa0940b15c0 2024-03-28T17:59:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.534+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa09415c070 con 0x7fa0940b15c0 2024-03-28T17:59:46.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.535+0000 7fa08e000640 1 -- 172.21.15.62:0/1919857113 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa08407b630 con 0x7fa0940b15c0 2024-03-28T17:59:46.535 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 msgr2=0x7fa07806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 0x7fa07806e090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa088007fd0 tx=0x7fa088009040 comp rx=0 tx=0).stop 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 msgr2=0x7fa0941570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa08402b910 tx=0x7fa0840046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 shutdown_connections 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa07806bbd0 0x7fa07806e090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0940b44f0 0x7fa0941f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0940b15c0 0x7fa0941570a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 --2- 172.21.15.62:0/1919857113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0940aee00 0x7fa0941500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.538+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 >> 172.21.15.62:0/1919857113 conn(0x7fa094066dd0 msgr2=0x7fa0940671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.539+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 shutdown_connections 2024-03-28T17:59:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.539+0000 7fa09b200640 1 -- 172.21.15.62:0/1919857113 wait complete. 2024-03-28T17:59:46.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:46.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.780+0000 7fb129e00640 1 Processor -- start 2024-03-28T17:59:46.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.781+0000 7fb129e00640 1 -- start start 2024-03-28T17:59:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.782+0000 7fb129e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb12415e2c0 0x7fb1240ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.782+0000 7fb129e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1240d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.782+0000 7fb129e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb124161bb0 con 0x7fb1240cec50 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.782+0000 7fb129e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb124161e90 con 0x7fb12415e2c0 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.782+0000 7fb129e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb124162170 con 0x7fb1240b3210 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb12415e2c0 0x7fb1240ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb12415e2c0 0x7fb1240ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32874/0 (socket says 172.21.15.62:32874) 2024-03-28T17:59:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 -- 172.21.15.62:0/2110287651 learned_addr learned my addr 172.21.15.62:0/2110287651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb129400640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1240d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 -- 172.21.15.62:0/2110287651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 msgr2=0x7fb1240d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1240d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 -- 172.21.15.62:0/2110287651 >> v1:172.21.15.62:6789/0 conn(0x7fb1240cec50 legacy=0x7fb1240cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123e00640 1 -- 172.21.15.62:0/2110287651 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124162450 con 0x7fb12415e2c0 2024-03-28T17:59:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.783+0000 7fb123400640 1 -- 172.21.15.62:0/2110287651 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330829134 0 0) 0x7fb124161bb0 con 0x7fb1240cec50 2024-03-28T17:59:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.784+0000 7fb123e00640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12415e2c0 0x7fb1240ce570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83104c3339de314 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb123400640 1 -- 172.21.15.62:0/2110287651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110034070 con 0x7fb12415e2c0 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb123400640 1 -- 172.21.15.62:0/2110287651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb110030840 con 0x7fb12415e2c0 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb123400640 1 -- 172.21.15.62:0/2110287651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110030c10 con 0x7fb12415e2c0 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 -- 172.21.15.62:0/2110287651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12415e2c0 msgr2=0x7fb1240ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12415e2c0 0x7fb1240ce570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002b980 comp rx=0 tx=0).stop 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 -- 172.21.15.62:0/2110287651 shutdown_connections 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1240d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 --2- 172.21.15.62:0/2110287651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12415e2c0 0x7fb1240ce570 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 -- 172.21.15.62:0/2110287651 >> 172.21.15.62:0/2110287651 conn(0x7fb124066fb0 msgr2=0x7fb1240673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 -- 172.21.15.62:0/2110287651 shutdown_connections 2024-03-28T17:59:46.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.785+0000 7fb129e00640 1 -- 172.21.15.62:0/2110287651 wait complete. 2024-03-28T17:59:46.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.787+0000 7fb129e00640 1 Processor -- start 2024-03-28T17:59:46.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.787+0000 7fb129e00640 1 -- start start 2024-03-28T17:59:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1241f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1240cec50 0x7fb124156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240d24f0 con 0x7fb1240ca3a0 2024-03-28T17:59:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124162a10 con 0x7fb1240cec50 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.788+0000 7fb129e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240d2670 con 0x7fb1240b3210 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32880/0 (socket says 172.21.15.62:32880) 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 -- 172.21.15.62:0/2033332425 learned_addr learned my addr 172.21.15.62:0/2033332425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb128a00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1241f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb123e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1240cec50 0x7fb124156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 -- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 msgr2=0x7fb1241f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1241f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 -- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1240cec50 msgr2=0x7fb124156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1240cec50 0x7fb124156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 -- 172.21.15.62:0/2033332425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240cca90 con 0x7fb1240ca3a0 2024-03-28T17:59:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.789+0000 7fb129400640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb11c007db0 tx=0x7fb11c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.790+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c014420 con 0x7fb1240ca3a0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.790+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c019070 con 0x7fb1240ca3a0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.790+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c014800 con 0x7fb1240ca3a0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.790+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1240ccd70 con 0x7fb1240ca3a0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.790+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1240cd330 con 0x7fb1240ca3a0 2024-03-28T17:59:46.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.791+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12404f400 con 0x7fb1240ca3a0 2024-03-28T17:59:46.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.792+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb11c0149a0 con 0x7fb1240ca3a0 2024-03-28T17:59:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.793+0000 7fb120c00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.793+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb11c08b400 con 0x7fb1240ca3a0 2024-03-28T17:59:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.793+0000 7fb128a00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:46.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.795+0000 7fb128a00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 0x7fb10406e090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb114000f60 tx=0x7fb114009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:46.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:46.798+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb11c059dd0 con 0x7fb1240ca3a0 2024-03-28T17:59:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.001+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb12415c480 con 0x7fb1240ca3a0 2024-03-28T17:59:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.002+0000 7fb120c00640 1 -- 172.21.15.62:0/2033332425 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb11c0596d0 con 0x7fb1240ca3a0 2024-03-28T17:59:47.003 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 msgr2=0x7fb10406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 0x7fb10406e090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb114000f60 tx=0x7fb114009040 comp rx=0 tx=0).stop 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 msgr2=0x7fb124157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb11c007db0 tx=0x7fb11c00e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 shutdown_connections 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1240ca3a0 0x7fb124157010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1240cec50 0x7fb124156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 --2- 172.21.15.62:0/2033332425 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1240b3210 0x7fb1241f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.005+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 >> 172.21.15.62:0/2033332425 conn(0x7fb124066fb0 msgr2=0x7fb12415cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.006+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 shutdown_connections 2024-03-28T17:59:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.006+0000 7fb129e00640 1 -- 172.21.15.62:0/2033332425 wait complete. 2024-03-28T17:59:47.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.249+0000 7f6fe0a00640 1 Processor -- start 2024-03-28T17:59:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.249+0000 7f6fe0a00640 1 -- start start 2024-03-28T17:59:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fe0a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 0x7f6fdc06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fe0a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6fdc06b8e0 0x7f6fdc162010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fe0a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0af820 con 0x7f6fdc0c5170 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fe0a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b2ee0 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fe0a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b22d0 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fd9e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6fdc06b8e0 0x7f6fdc162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fda800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 0x7f6fdc06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fd9e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6fdc06b8e0 0x7f6fdc162010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59502/0 (socket says 172.21.15.62:59502) 2024-03-28T17:59:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.251+0000 7f6fd9e00640 1 -- 172.21.15.62:0/4238286736 learned_addr learned my addr 172.21.15.62:0/4238286736 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.252+0000 7f6fd9e00640 1 -- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 msgr2=0x7f6fdc06b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.252+0000 7f6fd9e00640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 0x7f6fdc06b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.252+0000 7f6fd9e00640 1 -- 172.21.15.62:0/4238286736 >> v1:172.21.15.81:6789/0 conn(0x7f6fdc0c5170 legacy=0x7f6fdc166100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.252+0000 7f6fd9e00640 1 -- 172.21.15.62:0/4238286736 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc0b25b0 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.252+0000 7f6fda800640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 0x7f6fdc06b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.254+0000 7f6fd9e00640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc06b8e0 0x7f6fdc162010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009870 tx=0x7f6fc802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf970a4fac1ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.254+0000 7f6fd9400640 1 -- 172.21.15.62:0/4238286736 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8034070 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.254+0000 7f6fd9400640 1 -- 172.21.15.62:0/4238286736 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8030840 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.254+0000 7f6fd9400640 1 -- 172.21.15.62:0/4238286736 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8030c10 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 -- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc06b8e0 msgr2=0x7f6fdc162010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc06b8e0 0x7f6fdc162010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009870 tx=0x7f6fc802b980 comp rx=0 tx=0).stop 2024-03-28T17:59:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 -- 172.21.15.62:0/4238286736 shutdown_connections 2024-03-28T17:59:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc06b8e0 0x7f6fdc162010 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/4238286736 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc0c4d90 0x7f6fdc06b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 -- 172.21.15.62:0/4238286736 >> 172.21.15.62:0/4238286736 conn(0x7f6fdc0d1b50 msgr2=0x7f6fdc0d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:47.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 -- 172.21.15.62:0/4238286736 shutdown_connections 2024-03-28T17:59:47.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.255+0000 7f6fe0a00640 1 -- 172.21.15.62:0/4238286736 wait complete. 2024-03-28T17:59:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.257+0000 7f6fe0a00640 1 Processor -- start 2024-03-28T17:59:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.257+0000 7f6fe0a00640 1 -- start start 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc06b8e0 0x7f6fdc156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc0c5170 0x7f6fdc0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b3370 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b2b70 con 0x7f6fdc0c5170 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fe0a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b1e80 con 0x7f6fdc06b8e0 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32882/0 (socket says 172.21.15.62:32882) 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.258+0000 7f6fd9e00640 1 -- 172.21.15.62:0/3667582550 learned_addr learned my addr 172.21.15.62:0/3667582550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fda800640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc06b8e0 0x7f6fdc156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 -- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc06b8e0 msgr2=0x7f6fdc156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fdb200640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc0c5170 0x7f6fdc0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc06b8e0 0x7f6fdc156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 -- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc0c5170 msgr2=0x7f6fdc0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc0c5170 0x7f6fdc0c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 -- 172.21.15.62:0/3667582550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc15d170 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.259+0000 7f6fd9e00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.260+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8048070 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.260+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8030640 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.260+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc15d450 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.260+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8034040 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.260+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc15da10 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.261+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc09fb90 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.262+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6fc804b070 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.263+0000 7f6fd2a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 0x7f6fbc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.263+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6fc80aca30 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.264+0000 7f6fda800640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 0x7f6fbc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.265+0000 7f6fda800640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 0x7f6fbc06e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009bd0 tx=0x7f6fcc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.267+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6fc807b400 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.491+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fdc04f4e0 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.492+0000 7f6fd2a00640 1 -- 172.21.15.62:0/3667582550 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6fc8030810 con 0x7f6fdc0c4d90 2024-03-28T17:59:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 msgr2=0x7f6fbc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 0x7f6fbc06e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009bd0 tx=0x7f6fcc00b040 comp rx=0 tx=0).stop 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 msgr2=0x7f6fdc15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc8004730 comp rx=0 tx=0).stop 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 shutdown_connections 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6fbc06bbd0 0x7f6fbc06e090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fdc0c5170 0x7f6fdc0c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0c4d90 0x7f6fdc15e210 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 --2- 172.21.15.62:0/3667582550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6fdc06b8e0 0x7f6fdc156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 >> 172.21.15.62:0/3667582550 conn(0x7f6fdc0d1b50 msgr2=0x7f6fdc0d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 shutdown_connections 2024-03-28T17:59:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.495+0000 7f6fe0a00640 1 -- 172.21.15.62:0/3667582550 wait complete. 2024-03-28T17:59:47.514 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:47.514 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:47.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.801+0000 7f0a42000640 1 Processor -- start 2024-03-28T17:59:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.801+0000 7f0a42000640 1 -- start start 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a3c144740 0x7f0a3c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 0x7f0a3c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 0x7f0a3c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b3700 con 0x7f0a3c144740 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b39e0 con 0x7f0a3c15ed90 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.802+0000 7f0a42000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b3cc0 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a3c144740 0x7f0a3c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a3c144740 0x7f0a3c0b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32898/0 (socket says 172.21.15.62:32898) 2024-03-28T17:59:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 -- 172.21.15.62:0/984730398 learned_addr learned my addr 172.21.15.62:0/984730398 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3aa00640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 0x7f0a3c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3be00640 1 --2- 172.21.15.62:0/984730398 >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 0x7f0a3c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 -- 172.21.15.62:0/984730398 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 msgr2=0x7f0a3c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 0x7f0a3c15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 -- 172.21.15.62:0/984730398 >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 msgr2=0x7f0a3c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 --2- 172.21.15.62:0/984730398 >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 0x7f0a3c0ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3b400640 1 -- 172.21.15.62:0/984730398 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0b3fa0 con 0x7f0a3c144740 2024-03-28T17:59:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3aa00640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 0x7f0a3c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.803+0000 7f0a3be00640 1 --2- 172.21.15.62:0/984730398 >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 0x7f0a3c0ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.804+0000 7f0a3b400640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c144740 0x7f0a3c0b2cb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c15e790 tx=0x7f0a2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02bae6c26e460b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a3a000640 1 -- 172.21.15.62:0/984730398 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c035020 con 0x7f0a3c144740 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a3a000640 1 -- 172.21.15.62:0/984730398 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c031870 con 0x7f0a3c144740 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a3a000640 1 -- 172.21.15.62:0/984730398 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c031c40 con 0x7f0a3c144740 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 -- 172.21.15.62:0/984730398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c144740 msgr2=0x7f0a3c0b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c144740 0x7f0a3c0b2cb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c15e790 tx=0x7f0a2c02b910 comp rx=0 tx=0).stop 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 -- 172.21.15.62:0/984730398 shutdown_connections 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 --2- 172.21.15.62:0/984730398 >> v2:172.21.15.81:3300/0 conn(0x7f0a3c15ed90 0x7f0a3c0ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c0b31f0 0x7f0a3c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 --2- 172.21.15.62:0/984730398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c144740 0x7f0a3c0b2cb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.805+0000 7f0a42000640 1 -- 172.21.15.62:0/984730398 >> 172.21.15.62:0/984730398 conn(0x7f0a3c066fb0 msgr2=0x7f0a3c0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.806+0000 7f0a42000640 1 -- 172.21.15.62:0/984730398 shutdown_connections 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.806+0000 7f0a42000640 1 -- 172.21.15.62:0/984730398 wait complete. 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.807+0000 7f0a42000640 1 Processor -- start 2024-03-28T17:59:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.807+0000 7f0a42000640 1 -- start start 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c144740 0x7f0a3c137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c15ed90 0x7f0a3c13b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0aed40 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0aebc0 con 0x7f0a3c15ed90 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.808+0000 7f0a42000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0aeec0 con 0x7f0a3c144740 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32906/0 (socket says 172.21.15.62:32906) 2024-03-28T17:59:47.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 -- 172.21.15.62:0/2307277274 learned_addr learned my addr 172.21.15.62:0/2307277274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:47.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 -- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c144740 msgr2=0x7f0a3c137b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c144740 0x7f0a3c137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 -- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c15ed90 msgr2=0x7f0a3c13b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c15ed90 0x7f0a3c13b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 -- 172.21.15.62:0/2307277274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0b0460 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.809+0000 7f0a3b400640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c02b950 tx=0x7f0a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.810+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c040070 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.810+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c03a420 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.810+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c035040 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.810+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c0c9160 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.810+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c0c9720 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.811+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c0680c0 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.812+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0a2c04b060 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.813+0000 7f0a33400640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 0x7f0a1c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:47.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.813+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0a2c0aca40 con 0x7f0a3c0b31f0 2024-03-28T17:59:47.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.814+0000 7f0a3aa00640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 0x7f0a1c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:47.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.815+0000 7f0a3aa00640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 0x7f0a1c06e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c06cd30 tx=0x7f0a28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:47.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:47.818+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f0a2c07b540 con 0x7f0a3c0b31f0 2024-03-28T17:59:48.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.022+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a3c0b0940 con 0x7f0a3c0b31f0 2024-03-28T17:59:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.024+0000 7f0a33400640 1 -- 172.21.15.62:0/2307277274 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f0a2c0ace20 con 0x7f0a3c0b31f0 2024-03-28T17:59:48.025 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:48.025 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:48.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 msgr2=0x7f0a1c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 0x7f0a1c06e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c06cd30 tx=0x7f0a28008040 comp rx=0 tx=0).stop 2024-03-28T17:59:48.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 msgr2=0x7f0a3c0b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c02b950 tx=0x7f0a2c0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 shutdown_connections 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f0a1c06bbd0 0x7f0a1c06e090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c15ed90 0x7f0a3c13b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0a3c144740 0x7f0a3c137b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 --2- 172.21.15.62:0/2307277274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b31f0 0x7f0a3c0b0c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 >> 172.21.15.62:0/2307277274 conn(0x7f0a3c066fb0 msgr2=0x7f0a3c15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 shutdown_connections 2024-03-28T17:59:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.027+0000 7f0a42000640 1 -- 172.21.15.62:0/2307277274 wait complete. 2024-03-28T17:59:48.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:48.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.269+0000 7ff33a600640 1 Processor -- start 2024-03-28T17:59:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.269+0000 7ff33a600640 1 -- start start 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.270+0000 7ff33a600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff334070960 0x7ff3340b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.270+0000 7ff33a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3340cecd0 0x7ff3340d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.270+0000 7ff33a600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340cc7c0 con 0x7ff3340cd860 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.270+0000 7ff33a600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3340cd3d0 con 0x7ff3340cecd0 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.270+0000 7ff33a600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3340ccaa0 con 0x7ff334070960 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3340cecd0 0x7ff3340d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3340cecd0 0x7ff3340d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32914/0 (socket says 172.21.15.62:32914) 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 -- 172.21.15.62:0/2137369225 learned_addr learned my addr 172.21.15.62:0/2137369225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff332a00640 1 --1- 172.21.15.62:0/2137369225 >> v1:172.21.15.62:6789/0 conn(0x7ff3340cd860 0x7ff3340ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45346/0 (socket says 172.21.15.62:45346) 2024-03-28T17:59:48.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333400640 1 --2- 172.21.15.62:0/2137369225 >> v2:172.21.15.81:3300/0 conn(0x7ff334070960 0x7ff3340b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 -- 172.21.15.62:0/2137369225 >> v2:172.21.15.81:3300/0 conn(0x7ff334070960 msgr2=0x7ff3340b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 --2- 172.21.15.62:0/2137369225 >> v2:172.21.15.81:3300/0 conn(0x7ff334070960 0x7ff3340b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 -- 172.21.15.62:0/2137369225 >> v1:172.21.15.62:6789/0 conn(0x7ff3340cd860 legacy=0x7ff3340ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.271+0000 7ff333e00640 1 -- 172.21.15.62:0/2137369225 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340ccd80 con 0x7ff3340cecd0 2024-03-28T17:59:48.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.272+0000 7ff333e00640 1 --2- 172.21.15.62:0/2137369225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340d0640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff334071d30 tx=0x7ff32c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821985fa2af1f942 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.273+0000 7ff332000640 1 -- 172.21.15.62:0/2137369225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c034070 con 0x7ff3340cecd0 2024-03-28T17:59:48.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.273+0000 7ff332000640 1 -- 172.21.15.62:0/2137369225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c030870 con 0x7ff3340cecd0 2024-03-28T17:59:48.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.273+0000 7ff332000640 1 -- 172.21.15.62:0/2137369225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c030c40 con 0x7ff3340cecd0 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.273+0000 7ff33a600640 1 -- 172.21.15.62:0/2137369225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 msgr2=0x7ff3340d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.273+0000 7ff33a600640 1 --2- 172.21.15.62:0/2137369225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340d0640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff334071d30 tx=0x7ff32c02b9b0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 -- 172.21.15.62:0/2137369225 shutdown_connections 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 --2- 172.21.15.62:0/2137369225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340d0640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 --2- 172.21.15.62:0/2137369225 >> v2:172.21.15.81:3300/0 conn(0x7ff334070960 0x7ff3340b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 -- 172.21.15.62:0/2137369225 >> 172.21.15.62:0/2137369225 conn(0x7ff3340b3700 msgr2=0x7ff3340b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 -- 172.21.15.62:0/2137369225 shutdown_connections 2024-03-28T17:59:48.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.274+0000 7ff33a600640 1 -- 172.21.15.62:0/2137369225 wait complete. 2024-03-28T17:59:48.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.276+0000 7ff33a600640 1 Processor -- start 2024-03-28T17:59:48.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.276+0000 7ff33a600640 1 -- start start 2024-03-28T17:59:48.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff334070960 0x7ff334151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff3340cd860 0x7ff33406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33406ef20 con 0x7ff3340cecd0 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33406eda0 con 0x7ff334070960 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff33a600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33406f0a0 con 0x7ff3340cd860 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.277+0000 7ff333e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32920/0 (socket says 172.21.15.62:32920) 2024-03-28T17:59:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 -- 172.21.15.62:0/2721945527 learned_addr learned my addr 172.21.15.62:0/2721945527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:48.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333400640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff334070960 0x7ff334151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff332a00640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff3340cd860 0x7ff33406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 -- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff3340cd860 msgr2=0x7ff33406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff3340cd860 0x7ff33406c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 -- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff334070960 msgr2=0x7ff334151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff334070960 0x7ff334151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 -- 172.21.15.62:0/2721945527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33415c980 con 0x7ff3340cecd0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.278+0000 7ff333e00640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff32c02b9f0 tx=0x7ff32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.279+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c034040 con 0x7ff3340cecd0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.279+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c039ce0 con 0x7ff3340cecd0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.279+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c043610 con 0x7ff3340cecd0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.279+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3340b4760 con 0x7ff3340cecd0 2024-03-28T17:59:48.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.279+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3340b4a00 con 0x7ff3340cecd0 2024-03-28T17:59:48.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.280+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3340687d0 con 0x7ff3340cecd0 2024-03-28T17:59:48.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.281+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff32c037070 con 0x7ff3340cecd0 2024-03-28T17:59:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.282+0000 7ff32b400640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 0x7ff31006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.282+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff32c0acc80 con 0x7ff3340cecd0 2024-03-28T17:59:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.282+0000 7ff333400640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 0x7ff31006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.284+0000 7ff333400640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 0x7ff31006e0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff324006fd0 tx=0x7ff324008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.287+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff33415c980 con 0x7ff3340cecd0 2024-03-28T17:59:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.487+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3340b7b30 con 0x7ff3340cecd0 2024-03-28T17:59:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.488+0000 7ff32b400640 1 -- 172.21.15.62:0/2721945527 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff32c03e070 con 0x7ff3340cecd0 2024-03-28T17:59:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.490+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 msgr2=0x7ff31006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.490+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 0x7ff31006e0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff324006fd0 tx=0x7ff324008040 comp rx=0 tx=0).stop 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.490+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 msgr2=0x7ff3340ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.490+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff32c02b9f0 tx=0x7ff32c0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 shutdown_connections 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff31006bc00 0x7ff31006e0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3340cecd0 0x7ff3340ca780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff3340cd860 0x7ff33406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 --2- 172.21.15.62:0/2721945527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff334070960 0x7ff334151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 >> 172.21.15.62:0/2721945527 conn(0x7ff3340b3700 msgr2=0x7ff3340aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 shutdown_connections 2024-03-28T17:59:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.491+0000 7ff33a600640 1 -- 172.21.15.62:0/2721945527 wait complete. 2024-03-28T17:59:48.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.731+0000 7fe54de00640 1 Processor -- start 2024-03-28T17:59:48.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.732+0000 7fe54de00640 1 -- start start 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe54de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe548071dd0 0x7fe5480d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe54de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe54de00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480d3790 con 0x7fe5480b2f40 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe54de00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5480b71b0 con 0x7fe548071dd0 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe54de00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480b6ef0 con 0x7fe5480d1960 2024-03-28T17:59:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe53e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe548071dd0 0x7fe5480d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.733+0000 7fe53e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe548071dd0 0x7fe5480d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32924/0 (socket says 172.21.15.62:32924) 2024-03-28T17:59:48.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe53e000640 1 -- 172.21.15.62:0/3762798054 learned_addr learned my addr 172.21.15.62:0/3762798054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:48.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe547400640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe53e000640 1 -- 172.21.15.62:0/3762798054 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 msgr2=0x7fe54815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe53e000640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe53e000640 1 -- 172.21.15.62:0/3762798054 >> v1:172.21.15.81:6789/0 conn(0x7fe5480b2f40 legacy=0x7fe5480716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.734+0000 7fe53e000640 1 -- 172.21.15.62:0/3762798054 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480d2ef0 con 0x7fe548071dd0 2024-03-28T17:59:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.735+0000 7fe53e000640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe5480d2850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe530009870 tx=0x7fe53002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148a54d8b967e873 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe546000640 1 -- 172.21.15.62:0/3762798054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53002f840 con 0x7fe548071dd0 2024-03-28T17:59:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe546000640 1 -- 172.21.15.62:0/3762798054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53002f9e0 con 0x7fe548071dd0 2024-03-28T17:59:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe546000640 1 -- 172.21.15.62:0/3762798054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53002fdb0 con 0x7fe548071dd0 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 -- 172.21.15.62:0/3762798054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 msgr2=0x7fe5480d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe5480d2850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe530009870 tx=0x7fe53002ba20 comp rx=0 tx=0).stop 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 -- 172.21.15.62:0/3762798054 shutdown_connections 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 --2- 172.21.15.62:0/3762798054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe5480d2850 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 -- 172.21.15.62:0/3762798054 >> 172.21.15.62:0/3762798054 conn(0x7fe5480d3d90 msgr2=0x7fe5480d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.736+0000 7fe54de00640 1 -- 172.21.15.62:0/3762798054 shutdown_connections 2024-03-28T17:59:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.737+0000 7fe54de00640 1 -- 172.21.15.62:0/3762798054 wait complete. 2024-03-28T17:59:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.738+0000 7fe54de00640 1 Processor -- start 2024-03-28T17:59:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.738+0000 7fe54de00640 1 -- start start 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 0x7fe5481e8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480d4bc0 con 0x7fe548071dd0 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480d34b0 con 0x7fe5480b2f40 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe54de00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480b3320 con 0x7fe5480d1960 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.739+0000 7fe546a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32936/0 (socket says 172.21.15.62:32936) 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 -- 172.21.15.62:0/1413638624 learned_addr learned my addr 172.21.15.62:0/1413638624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe547400640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe53e000640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 0x7fe5481e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 -- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 msgr2=0x7fe54815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 -- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 msgr2=0x7fe5481e8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 0x7fe5481e8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe546a00640 1 -- 172.21.15.62:0/1413638624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54815db70 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.740+0000 7fe53e000640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 0x7fe5481e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe546a00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c400 tx=0x7fe53800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380052d0 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538005490 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538007260 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5481f3cc0 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.741+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5481e8fe0 con 0x7fe548071dd0 2024-03-28T17:59:48.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.742+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5480680c0 con 0x7fe548071dd0 2024-03-28T17:59:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.743+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe538007400 con 0x7fe548071dd0 2024-03-28T17:59:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.744+0000 7fe53fe00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 0x7fe52406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.744+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe53808a100 con 0x7fe548071dd0 2024-03-28T17:59:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.745+0000 7fe53e000640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 0x7fe52406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.746+0000 7fe53e000640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 0x7fe52406e090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe530036000 tx=0x7fe530039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.749+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe538058ad0 con 0x7fe548071dd0 2024-03-28T17:59:48.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.949+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5481e91b0 con 0x7fe548071dd0 2024-03-28T17:59:48.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.950+0000 7fe53fe00640 1 -- 172.21.15.62:0/1413638624 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5380583d0 con 0x7fe548071dd0 2024-03-28T17:59:48.950 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 msgr2=0x7fe52406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 0x7fe52406e090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe530036000 tx=0x7fe530039040 comp rx=0 tx=0).stop 2024-03-28T17:59:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 msgr2=0x7fe54815ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c400 tx=0x7fe53800c8c0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 shutdown_connections 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe52406bbd0 0x7fe52406e090 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe5480d1960 0x7fe54815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5480b2f40 0x7fe5481e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 --2- 172.21.15.62:0/1413638624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe548071dd0 0x7fe54815ddc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 >> 172.21.15.62:0/1413638624 conn(0x7fe5480d3d90 msgr2=0x7fe5480b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 shutdown_connections 2024-03-28T17:59:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:48.953+0000 7fe54de00640 1 -- 172.21.15.62:0/1413638624 wait complete. 2024-03-28T17:59:48.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:49.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.185+0000 7f467f200640 1 Processor -- start 2024-03-28T17:59:49.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.186+0000 7f467f200640 1 -- start start 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46800aee00 0x7f46800b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4680160190 0x7f468015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46800b44f0 0x7f46800b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800b4e80 con 0x7f4680160190 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468015cfd0 con 0x7f46800aee00 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.187+0000 7f467f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468015bec0 con 0x7f46800b44f0 2024-03-28T17:59:49.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4680160190 0x7f468015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4680160190 0x7f468015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32940/0 (socket says 172.21.15.62:32940) 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 -- 172.21.15.62:0/4291976349 learned_addr learned my addr 172.21.15.62:0/4291976349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f467de00640 1 --2- 172.21.15.62:0/4291976349 >> v2:172.21.15.81:3300/0 conn(0x7f46800aee00 0x7f46800b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f467e800640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46800b44f0 0x7f46800b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 -- 172.21.15.62:0/4291976349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46800b44f0 msgr2=0x7f46800b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46800b44f0 0x7f46800b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 -- 172.21.15.62:0/4291976349 >> v2:172.21.15.81:3300/0 conn(0x7f46800aee00 msgr2=0x7f46800b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:49.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 --2- 172.21.15.62:0/4291976349 >> v2:172.21.15.81:3300/0 conn(0x7f46800aee00 0x7f46800b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.188+0000 7f4677e00640 1 -- 172.21.15.62:0/4291976349 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800b1bf0 con 0x7f4680160190 2024-03-28T17:59:49.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.190+0000 7f4677e00640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4680160190 0x7f468015ca90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f468006e9a0 tx=0x7f466802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4557382dad339a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.190+0000 7f467d400640 1 -- 172.21.15.62:0/4291976349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668035020 con 0x7f4680160190 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.190+0000 7f467d400640 1 -- 172.21.15.62:0/4291976349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668031870 con 0x7f4680160190 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.190+0000 7f467d400640 1 -- 172.21.15.62:0/4291976349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668031c40 con 0x7f4680160190 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 -- 172.21.15.62:0/4291976349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4680160190 msgr2=0x7f468015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4680160190 0x7f468015ca90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f468006e9a0 tx=0x7f466802b910 comp rx=0 tx=0).stop 2024-03-28T17:59:49.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 -- 172.21.15.62:0/4291976349 shutdown_connections 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f46800b44f0 0x7f46800b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 --2- 172.21.15.62:0/4291976349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4680160190 0x7f468015ca90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 --2- 172.21.15.62:0/4291976349 >> v2:172.21.15.81:3300/0 conn(0x7f46800aee00 0x7f46800b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 -- 172.21.15.62:0/4291976349 >> 172.21.15.62:0/4291976349 conn(0x7f4680066dd0 msgr2=0x7f46800671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 -- 172.21.15.62:0/4291976349 shutdown_connections 2024-03-28T17:59:49.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.191+0000 7f467f200640 1 -- 172.21.15.62:0/4291976349 wait complete. 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.193+0000 7f467f200640 1 Processor -- start 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.193+0000 7f467f200640 1 -- start start 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46800aee00 0x7f468014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4680160190 0x7f4680152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015f990 con 0x7f46800b44f0 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015f6b0 con 0x7f46800aee00 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468015fc70 con 0x7f4680160190 2024-03-28T17:59:49.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32944/0 (socket says 172.21.15.62:32944) 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 -- 172.21.15.62:0/1563258215 learned_addr learned my addr 172.21.15.62:0/1563258215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467de00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46800aee00 0x7f468014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f467e800640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4680160190 0x7f4680152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 -- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4680160190 msgr2=0x7f4680152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4680160190 0x7f4680152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 -- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46800aee00 msgr2=0x7f468014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.194+0000 7f4677e00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46800aee00 0x7f468014aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f4677e00640 1 -- 172.21.15.62:0/1563258215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46801535a0 con 0x7f46800b44f0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f4677e00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f466802b950 tx=0x7f46680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668040070 con 0x7f46800b44f0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668030c80 con 0x7f46800b44f0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668035040 con 0x7f46800b44f0 2024-03-28T17:59:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4680153880 con 0x7f46800b44f0 2024-03-28T17:59:49.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.195+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46800af740 con 0x7f46800b44f0 2024-03-28T17:59:49.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.197+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680153da0 con 0x7f46800b44f0 2024-03-28T17:59:49.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.197+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f466804b060 con 0x7f46800b44f0 2024-03-28T17:59:49.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.198+0000 7f4676000640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 0x7f465c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.198+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f46680ac940 con 0x7f46800b44f0 2024-03-28T17:59:49.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.199+0000 7f467de00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 0x7f465c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.200+0000 7f467de00640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 0x7f465c06e090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4670006fd0 tx=0x7f4670008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.204+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4680153da0 con 0x7f46800b44f0 2024-03-28T17:59:49.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.411+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f468004f400 con 0x7f46800b44f0 2024-03-28T17:59:49.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.412+0000 7f4676000640 1 -- 172.21.15.62:0/1563258215 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f466802f320 con 0x7f46800b44f0 2024-03-28T17:59:49.412 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 msgr2=0x7f465c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 0x7f465c06e090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4670006fd0 tx=0x7f4670008040 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 msgr2=0x7f46801f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f466802b950 tx=0x7f46680046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 shutdown_connections 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f465c06bbd0 0x7f465c06e090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4680160190 0x7f4680152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46800b44f0 0x7f46801f1460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 --2- 172.21.15.62:0/1563258215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46800aee00 0x7f468014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 >> 172.21.15.62:0/1563258215 conn(0x7f4680066dd0 msgr2=0x7f46800cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 shutdown_connections 2024-03-28T17:59:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.415+0000 7f467f200640 1 -- 172.21.15.62:0/1563258215 wait complete. 2024-03-28T17:59:49.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.656+0000 7f58dc200640 1 Processor -- start 2024-03-28T17:59:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.656+0000 7f58dc200640 1 -- start start 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.657+0000 7f58dc200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d40719b0 0x7f58d40b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.657+0000 7f58dc200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f58d40ae370 0x7f58d40ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58dc200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d40b30d0 con 0x7f58d4071d90 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58dc200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d40b33b0 con 0x7f58d40719b0 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58dc200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d40b3690 con 0x7f58d40ae370 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d40719b0 0x7f58d40b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d40719b0 0x7f58d40b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32958/0 (socket says 172.21.15.62:32958) 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 -- 172.21.15.62:0/3000375363 learned_addr learned my addr 172.21.15.62:0/3000375363 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d8e00640 1 --2- 172.21.15.62:0/3000375363 >> v2:172.21.15.81:3300/0 conn(0x7f58d40ae370 0x7f58d40ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 -- 172.21.15.62:0/3000375363 >> v2:172.21.15.81:3300/0 conn(0x7f58d40ae370 msgr2=0x7f58d40ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 --2- 172.21.15.62:0/3000375363 >> v2:172.21.15.81:3300/0 conn(0x7f58d40ae370 0x7f58d40ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 -- 172.21.15.62:0/3000375363 >> v1:172.21.15.81:6789/0 conn(0x7f58d4071d90 legacy=0x7f58d4072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.658+0000 7f58d9800640 1 -- 172.21.15.62:0/3000375363 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40b3970 con 0x7f58d40719b0 2024-03-28T17:59:49.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.660+0000 7f58d9800640 1 --2- 172.21.15.62:0/3000375363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40719b0 0x7f58d40b4170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58d406ec60 tx=0x7f58c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f938c754186d6c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.660+0000 7f58d3e00640 1 -- 172.21.15.62:0/3000375363 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8035020 con 0x7f58d40719b0 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.660+0000 7f58d3e00640 1 -- 172.21.15.62:0/3000375363 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8031870 con 0x7f58d40719b0 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.660+0000 7f58d3e00640 1 -- 172.21.15.62:0/3000375363 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8031c40 con 0x7f58d40719b0 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 -- 172.21.15.62:0/3000375363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40719b0 msgr2=0x7f58d40b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 --2- 172.21.15.62:0/3000375363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40719b0 0x7f58d40b4170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58d406ec60 tx=0x7f58c802b910 comp rx=0 tx=0).stop 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 -- 172.21.15.62:0/3000375363 shutdown_connections 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 --2- 172.21.15.62:0/3000375363 >> v2:172.21.15.81:3300/0 conn(0x7f58d40ae370 0x7f58d40ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 --2- 172.21.15.62:0/3000375363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40719b0 0x7f58d40b4170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 -- 172.21.15.62:0/3000375363 >> 172.21.15.62:0/3000375363 conn(0x7f58d4066fb0 msgr2=0x7f58d40673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 -- 172.21.15.62:0/3000375363 shutdown_connections 2024-03-28T17:59:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.661+0000 7f58dc200640 1 -- 172.21.15.62:0/3000375363 wait complete. 2024-03-28T17:59:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.663+0000 7f58dc200640 1 Processor -- start 2024-03-28T17:59:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.663+0000 7f58dc200640 1 -- start start 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 0x7f58d41f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f58d4071d90 0x7f58d414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40b46b0 con 0x7f58d40ae370 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40aed10 con 0x7f58d40719b0 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58dc200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d406ccb0 con 0x7f58d4071d90 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58da200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58da200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32972/0 (socket says 172.21.15.62:32972) 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58da200640 1 -- 172.21.15.62:0/3337181501 learned_addr learned my addr 172.21.15.62:0/3337181501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58d9800640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 0x7f58d41f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.664+0000 7f58d8e00640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f58d4071d90 0x7f58d414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 -- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f58d4071d90 msgr2=0x7f58d414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f58d4071d90 0x7f58d414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 -- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 msgr2=0x7f58d41f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 0x7f58d41f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 -- 172.21.15.62:0/3337181501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d409ab70 con 0x7f58d40ae370 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58d9800640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 0x7f58d41f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58da200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c780 tx=0x7f58cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc017070 con 0x7f58d40ae370 2024-03-28T17:59:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc003da0 con 0x7f58d40ae370 2024-03-28T17:59:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0052b0 con 0x7f58d40ae370 2024-03-28T17:59:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.665+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d409ae50 con 0x7f58d40ae370 2024-03-28T17:59:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.666+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d409b410 con 0x7f58d40ae370 2024-03-28T17:59:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.667+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d40680c0 con 0x7f58d40ae370 2024-03-28T17:59:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.668+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f58cc01a070 con 0x7f58d40ae370 2024-03-28T17:59:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.669+0000 7f58d1600640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 0x7f58b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.669+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f58cc08a110 con 0x7f58d40ae370 2024-03-28T17:59:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.670+0000 7f58d9800640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 0x7f58b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.671+0000 7f58d9800640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 0x7f58b806e090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58c8036000 tx=0x7f58c803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.673+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f58cc058c10 con 0x7f58d40ae370 2024-03-28T17:59:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.887+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58d406bbb0 con 0x7f58d40ae370 2024-03-28T17:59:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.887+0000 7f58d1600640 1 -- 172.21.15.62:0/3337181501 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f58cc058510 con 0x7f58d40ae370 2024-03-28T17:59:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 msgr2=0x7f58b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 0x7f58b806e090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58c8036000 tx=0x7f58c803c040 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 msgr2=0x7f58d409bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c780 tx=0x7f58cc00cc40 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 shutdown_connections 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f58b806bbd0 0x7f58b806e090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d40ae370 0x7f58d409bc10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f58d4071d90 0x7f58d414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 --2- 172.21.15.62:0/3337181501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d40719b0 0x7f58d41f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 >> 172.21.15.62:0/3337181501 conn(0x7f58d4066fb0 msgr2=0x7f58d40b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 shutdown_connections 2024-03-28T17:59:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:49.890+0000 7f58dc200640 1 -- 172.21.15.62:0/3337181501 wait complete. 2024-03-28T17:59:49.910 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-03-28T17:59:49.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:49.910 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-28T17:59:50.010 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T17:59:50.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T17:59:50.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:50.011 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-28T17:59:50.076 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:50.076 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:50.135 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:50.135 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:50.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.405+0000 7ff058e00640 1 Processor -- start 2024-03-28T17:59:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.406+0000 7ff058e00640 1 -- start start 2024-03-28T17:59:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff058e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0540d2960 0x7ff0540b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff058e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540d1900 con 0x7ff0540b5c50 2024-03-28T17:59:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff058e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540d2df0 con 0x7ff05415de10 2024-03-28T17:59:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff058e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540b60a0 con 0x7ff0540d2960 2024-03-28T17:59:50.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff053e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff0540b5c50 0x7ff0540b6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45358/0 (socket says 172.21.15.62:45358) 2024-03-28T17:59:50.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff053e00640 1 -- 172.21.15.62:0/1670536989 learned_addr learned my addr 172.21.15.62:0/1670536989 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:50.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.407+0000 7ff052a00640 1 --2- 172.21.15.62:0/1670536989 >> v2:172.21.15.81:3300/0 conn(0x7ff0540d2960 0x7ff0540b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020367220 0 0) 0x7ff0540d1900 con 0x7ff0540b5c50 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff038003520 con 0x7ff0540b5c50 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563153083 0 0) 0x7ff038003520 con 0x7ff0540b5c50 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 >> v2:172.21.15.81:3300/0 conn(0x7ff0540d2960 msgr2=0x7ff0540b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 --2- 172.21.15.62:0/1670536989 >> v2:172.21.15.81:3300/0 conn(0x7ff0540d2960 0x7ff0540b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 >> v1:172.21.15.81:6789/0 conn(0x7ff05415de10 legacy=0x7ff0540b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540d1ba0 con 0x7ff0540b5c50 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7ff04c001690 con 0x7ff0540b5c50 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 >> v1:172.21.15.62:6789/0 conn(0x7ff0540b5c50 legacy=0x7ff0540b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380039b0 0x7ff038003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff038004800 0x7ff038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 0x7ff03800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540b60a0 con 0x7ff0380051c0 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03800c070 con 0x7ff038004800 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.408+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03800c270 con 0x7ff0380039b0 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 0x7ff03800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053400640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380039b0 0x7ff038003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff052a00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff038004800 0x7ff038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 -- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380039b0 msgr2=0x7ff038003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380039b0 0x7ff038003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 -- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff038004800 msgr2=0x7ff038004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff038004800 0x7ff038004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.409+0000 7ff053e00640 1 -- 172.21.15.62:0/1670536989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038003520 con 0x7ff0380051c0 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.410+0000 7ff053e00640 1 -- 172.21.15.62:0/1670536989 reap_dead start 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.410+0000 7ff053e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 0x7ff03800b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff04c007fd0 tx=0x7ff038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98adc6ef0f0a363d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.410+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c007d50 con 0x7ff0380051c0 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.410+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c030070 con 0x7ff0380051c0 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff052000640 1 -- 172.21.15.62:0/1670536989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c005d30 con 0x7ff0380051c0 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 -- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 msgr2=0x7ff03800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 0x7ff03800b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff04c007fd0 tx=0x7ff038005830 comp rx=0 tx=0).stop 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 -- 172.21.15.62:0/1670536989 shutdown_connections 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 --2- 172.21.15.62:0/1670536989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0380051c0 0x7ff03800b9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 -- 172.21.15.62:0/1670536989 >> 172.21.15.62:0/1670536989 conn(0x7ff054071ae0 msgr2=0x7ff054071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 -- 172.21.15.62:0/1670536989 shutdown_connections 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.411+0000 7ff058e00640 1 -- 172.21.15.62:0/1670536989 wait complete. 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.413+0000 7ff058e00640 1 Processor -- start 2024-03-28T17:59:50.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.413+0000 7ff058e00640 1 -- start start 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380051c0 0x7ff054157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff05414dc80 0x7ff0540c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540d2160 con 0x7ff054159090 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540d24d0 con 0x7ff05414dc80 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff058e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05415e1f0 con 0x7ff0380051c0 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff053e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff053e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32996/0 (socket says 172.21.15.62:32996) 2024-03-28T17:59:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.414+0000 7ff053e00640 1 -- 172.21.15.62:0/646081272 learned_addr learned my addr 172.21.15.62:0/646081272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff052a00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff05414dc80 0x7ff0540c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053400640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380051c0 0x7ff054157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 -- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380051c0 msgr2=0x7ff054157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380051c0 0x7ff054157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 -- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff05414dc80 msgr2=0x7ff0540c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff05414dc80 0x7ff0540c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 -- 172.21.15.62:0/646081272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540ca1a0 con 0x7ff054159090 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.415+0000 7ff053e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000f60 tx=0x7ff04c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.416+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c045070 con 0x7ff054159090 2024-03-28T17:59:50.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.416+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c004c30 con 0x7ff054159090 2024-03-28T17:59:50.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.416+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c007d50 con 0x7ff054159090 2024-03-28T17:59:50.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.416+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff054157ff0 con 0x7ff054159090 2024-03-28T17:59:50.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.416+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0541585b0 con 0x7ff054159090 2024-03-28T17:59:50.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.417+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0540d0380 con 0x7ff054159090 2024-03-28T17:59:50.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.418+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff04c007070 con 0x7ff054159090 2024-03-28T17:59:50.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.419+0000 7ff04b400640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 0x7ff03006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.419+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff04c0abc00 con 0x7ff054159090 2024-03-28T17:59:50.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.419+0000 7ff053400640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 0x7ff03006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.421+0000 7ff053400640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 0x7ff03006e090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff044003f30 tx=0x7ff044009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.424+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff0540d0380 con 0x7ff054159090 2024-03-28T17:59:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.628+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0540c6f60 con 0x7ff054159090 2024-03-28T17:59:50.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.629+0000 7ff04b400640 1 -- 172.21.15.62:0/646081272 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7ff0540c6f60 con 0x7ff054159090 2024-03-28T17:59:50.630 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:50.631 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.632+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 msgr2=0x7ff03006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.632+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 0x7ff03006e090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff044003f30 tx=0x7ff044009040 comp rx=0 tx=0).stop 2024-03-28T17:59:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 msgr2=0x7ff0540b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000f60 tx=0x7ff04c001a60 comp rx=0 tx=0).stop 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 shutdown_connections 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff03006bbd0 0x7ff03006e090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff054159090 0x7ff0540b4b20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff05414dc80 0x7ff0540c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 --2- 172.21.15.62:0/646081272 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0380051c0 0x7ff054157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 >> 172.21.15.62:0/646081272 conn(0x7ff054071ae0 msgr2=0x7ff05415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 shutdown_connections 2024-03-28T17:59:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.633+0000 7ff058e00640 1 -- 172.21.15.62:0/646081272 wait complete. 2024-03-28T17:59:50.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:50.654 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.879+0000 7f91b8800640 1 Processor -- start 2024-03-28T17:59:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.880+0000 7f91b8800640 1 -- start start 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.881+0000 7f91b8800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91b00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.881+0000 7f91b8800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f91b0072ad0 0x7f91b00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b8800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b015d1b0 con 0x7f91b00ae720 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b8800640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b0070d50 con 0x7f91b0072ad0 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b8800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0071030 con 0x7f91b0072690 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b5e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f91b00ae720 0x7f91b00aeb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45368/0 (socket says 172.21.15.62:45368) 2024-03-28T17:59:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b5e00640 1 -- 172.21.15.62:0/864310924 learned_addr learned my addr 172.21.15.62:0/864310924 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b6800640 1 --2- 172.21.15.62:0/864310924 >> v2:172.21.15.81:3300/0 conn(0x7f91b0072ad0 0x7f91b00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b5400640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91b00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024803458 0 0) 0x7f91b015d1b0 con 0x7f91b00ae720 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.882+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9198003520 con 0x7f91b00ae720 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342994485 0 0) 0x7f9198003520 con 0x7f91b00ae720 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 msgr2=0x7f91b00cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91b00cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 >> v2:172.21.15.81:3300/0 conn(0x7f91b0072ad0 msgr2=0x7f91b00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 --2- 172.21.15.62:0/864310924 >> v2:172.21.15.81:3300/0 conn(0x7f91b0072ad0 0x7f91b00ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0071310 con 0x7f91b00ae720 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f91a4001670 con 0x7f91b00ae720 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 >> v1:172.21.15.62:6789/0 conn(0x7f91b00ae720 legacy=0x7f91b00aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91980039b0 0x7f9198003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198004800 0x7f9198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0070d50 con 0x7f9198004800 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0071030 con 0x7f91980039b0 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919800b9c0 con 0x7f91b0072690 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.883+0000 7f91b5400640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198004800 0x7f9198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b6800640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5e00640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91980039b0 0x7f9198003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5400640 1 -- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 msgr2=0x7f91980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5400640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5400640 1 -- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91980039b0 msgr2=0x7f9198003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5400640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91980039b0 0x7f9198003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b5400640 1 -- 172.21.15.62:0/864310924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198003520 con 0x7f9198004800 2024-03-28T17:59:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.884+0000 7f91b6800640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0072690 0x7f91980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.885+0000 7f91b5400640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198004800 0x7f9198004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009ac0 tx=0x7f9198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f566d52091720b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.885+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0039070 con 0x7f9198004800 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.885+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a002c6b0 con 0x7f9198004800 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.885+0000 7f91b4a00640 1 -- 172.21.15.62:0/864310924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0009690 con 0x7f9198004800 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 -- 172.21.15.62:0/864310924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198004800 msgr2=0x7f9198004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 --2- 172.21.15.62:0/864310924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9198004800 0x7f9198004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009ac0 tx=0x7f9198005830 comp rx=0 tx=0).stop 2024-03-28T17:59:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b6800640 1 -- 172.21.15.62:0/864310924 reap_dead start 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 -- 172.21.15.62:0/864310924 shutdown_connections 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 -- 172.21.15.62:0/864310924 >> 172.21.15.62:0/864310924 conn(0x7f91b00b1650 msgr2=0x7f91b00b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 -- 172.21.15.62:0/864310924 shutdown_connections 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.886+0000 7f91b8800640 1 -- 172.21.15.62:0/864310924 wait complete. 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.888+0000 7f91b8800640 1 Processor -- start 2024-03-28T17:59:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.888+0000 7f91b8800640 1 -- start start 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9198004800 0x7f91b01e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0142400 0x7f91b015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0070a40 con 0x7f91b0142880 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00718d0 con 0x7f9198004800 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b8800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0153030 con 0x7f91b0142400 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33028/0 (socket says 172.21.15.62:33028) 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.889+0000 7f91b6800640 1 -- 172.21.15.62:0/3017770683 learned_addr learned my addr 172.21.15.62:0/3017770683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b5e00640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9198004800 0x7f91b01e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b5400640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0142400 0x7f91b015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 -- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0142400 msgr2=0x7f91b015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0142400 0x7f91b015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 -- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9198004800 msgr2=0x7f91b01e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9198004800 0x7f91b01e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 -- 172.21.15.62:0/3017770683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b014e460 con 0x7f91b0142880 2024-03-28T17:59:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91b6800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91b006c7f0 tx=0x7f91ac00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac016620 con 0x7f91b0142880 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.890+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac01b070 con 0x7f91b0142880 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.891+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0169e0 con 0x7f91b0142880 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.891+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b014e740 con 0x7f91b0142880 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.891+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b014ed00 con 0x7f91b0142880 2024-03-28T17:59:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.892+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b00680c0 con 0x7f91b0142880 2024-03-28T17:59:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.893+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f91ac005d10 con 0x7f91b0142880 2024-03-28T17:59:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.894+0000 7f91aa000640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 0x7f919406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.894+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f91ac08b700 con 0x7f91b0142880 2024-03-28T17:59:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.894+0000 7f91b5e00640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 0x7f919406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.896+0000 7f91b5e00640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 0x7f919406e090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91a4001200 tx=0x7f91a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:50.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:50.899+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f91ac025020 con 0x7f91b0142880 2024-03-28T17:59:51.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.104+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91b00b2420 con 0x7f91b0142880 2024-03-28T17:59:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.105+0000 7f91aa000640 1 -- 172.21.15.62:0/3017770683 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91ac05a160 con 0x7f91b0142880 2024-03-28T17:59:51.105 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.107+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 msgr2=0x7f919406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.107+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 0x7f919406e090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91a4001200 tx=0x7f91a4009040 comp rx=0 tx=0).stop 2024-03-28T17:59:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.107+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 msgr2=0x7f91b015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.107+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91b006c7f0 tx=0x7f91ac00f520 comp rx=0 tx=0).stop 2024-03-28T17:59:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.107+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 shutdown_connections 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f919406bbd0 0x7f919406e090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b0142880 0x7f91b015d6f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f91b0142400 0x7f91b015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 --2- 172.21.15.62:0/3017770683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9198004800 0x7f91b01e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 >> 172.21.15.62:0/3017770683 conn(0x7f91b00b1650 msgr2=0x7f91b00ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 shutdown_connections 2024-03-28T17:59:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.108+0000 7f91b8800640 1 -- 172.21.15.62:0/3017770683 wait complete. 2024-03-28T17:59:51.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:51.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.349+0000 7f7939800640 1 Processor -- start 2024-03-28T17:59:51.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.350+0000 7f7939800640 1 -- start start 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.351+0000 7f7939800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79340b4c00 0x7f7934160190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.351+0000 7f7939800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793415bec0 con 0x7f79340b15c0 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.351+0000 7f7939800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79340b1bf0 con 0x7f79340aee00 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.351+0000 7f7939800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340b1ed0 con 0x7f79340b4c00 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f7938e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79340b4c00 0x7f7934160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f792be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f79340b15c0 0x7f79340b44f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45378/0 (socket says 172.21.15.62:45378) 2024-03-28T17:59:51.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f792be00640 1 -- 172.21.15.62:0/2386913365 learned_addr learned my addr 172.21.15.62:0/2386913365 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f7938e00640 1 -- 172.21.15.62:0/2386913365 >> v1:172.21.15.81:6789/0 conn(0x7f79340aee00 legacy=0x7f793415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f7938e00640 1 -- 172.21.15.62:0/2386913365 >> v1:172.21.15.62:6789/0 conn(0x7f79340b15c0 legacy=0x7f79340b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.352+0000 7f7938e00640 1 -- 172.21.15.62:0/2386913365 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793415f6b0 con 0x7f79340b4c00 2024-03-28T17:59:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.353+0000 7f7938e00640 1 --2- 172.21.15.62:0/2386913365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f7934160190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f792c009d90 tx=0x7f792c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77acccbe04ba2bec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7933400640 1 -- 172.21.15.62:0/2386913365 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c039070 con 0x7f79340b4c00 2024-03-28T17:59:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7933400640 1 -- 172.21.15.62:0/2386913365 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c02cc60 con 0x7f79340b4c00 2024-03-28T17:59:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7933400640 1 -- 172.21.15.62:0/2386913365 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c034660 con 0x7f79340b4c00 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 -- 172.21.15.62:0/2386913365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 msgr2=0x7f7934160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 --2- 172.21.15.62:0/2386913365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f7934160190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f792c009d90 tx=0x7f792c02beb0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 -- 172.21.15.62:0/2386913365 shutdown_connections 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 --2- 172.21.15.62:0/2386913365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f7934160190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 -- 172.21.15.62:0/2386913365 >> 172.21.15.62:0/2386913365 conn(0x7f7934066dd0 msgr2=0x7f79340671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 -- 172.21.15.62:0/2386913365 shutdown_connections 2024-03-28T17:59:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.354+0000 7f7939800640 1 -- 172.21.15.62:0/2386913365 wait complete. 2024-03-28T17:59:51.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.356+0000 7f7939800640 1 Processor -- start 2024-03-28T17:59:51.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.356+0000 7f7939800640 1 -- start start 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 0x7f793414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 0x7f793415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340af1e0 con 0x7f79340b4c00 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793415fc70 con 0x7f79340b15c0 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7939800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340b2930 con 0x7f79340aee00 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.357+0000 7f7938e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33048/0 (socket says 172.21.15.62:33048) 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 -- 172.21.15.62:0/1886298355 learned_addr learned my addr 172.21.15.62:0/1886298355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7933e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 0x7f793414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f792be00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 0x7f793415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 -- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 msgr2=0x7f793414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 0x7f793414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 -- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 msgr2=0x7f793415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 0x7f793415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 -- 172.21.15.62:0/1886298355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793414ca60 con 0x7f79340b4c00 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f792be00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 0x7f793415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7933e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 0x7f793414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7938e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f792c037040 tx=0x7f792c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.358+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c039050 con 0x7f79340b4c00 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.359+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c03a040 con 0x7f79340b4c00 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.359+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c03e610 con 0x7f79340b4c00 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.359+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79341e6140 con 0x7f79340b4c00 2024-03-28T17:59:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.359+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79341e6700 con 0x7f79340b4c00 2024-03-28T17:59:51.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.360+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79340b2500 con 0x7f79340b4c00 2024-03-28T17:59:51.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.361+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f792c047020 con 0x7f79340b4c00 2024-03-28T17:59:51.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.362+0000 7f7930c00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 0x7f791006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.362+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f792c0acf70 con 0x7f79340b4c00 2024-03-28T17:59:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.362+0000 7f7933e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 0x7f791006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.363+0000 7f7933e00640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 0x7f791006e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7924003eb0 tx=0x7f7924009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.367+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f79340b2500 con 0x7f79340b4c00 2024-03-28T17:59:51.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.571+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f793415cb80 con 0x7f79340b4c00 2024-03-28T17:59:51.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.572+0000 7f7930c00640 1 -- 172.21.15.62:0/1886298355 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f792c04e020 con 0x7f79340b4c00 2024-03-28T17:59:51.573 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:51.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 msgr2=0x7f791006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 0x7f791006e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7924003eb0 tx=0x7f7924009040 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 msgr2=0x7f793415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f792c037040 tx=0x7f792c004a40 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 shutdown_connections 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f791006bbd0 0x7f791006e090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79340b4c00 0x7f793415e750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79340b15c0 0x7f793415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 --2- 172.21.15.62:0/1886298355 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f79340aee00 0x7f793414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 >> 172.21.15.62:0/1886298355 conn(0x7f7934066dd0 msgr2=0x7f79340ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 shutdown_connections 2024-03-28T17:59:51.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.575+0000 7f7939800640 1 -- 172.21.15.62:0/1886298355 wait complete. 2024-03-28T17:59:51.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.823+0000 7f2fe3e00640 1 Processor -- start 2024-03-28T17:59:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.823+0000 7f2fe3e00640 1 -- start start 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe3e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fe416b030 0x7f2fe416d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe3e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe41628a0 con 0x7f2fe40b15c0 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe3e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe4162b80 con 0x7f2fe4070890 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe3e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4162e60 con 0x7f2fe416b030 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2fe40b15c0 0x7f2fe40b19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45390/0 (socket says 172.21.15.62:45390) 2024-03-28T17:59:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.825+0000 7f2fe2a00640 1 -- 172.21.15.62:0/3814911117 learned_addr learned my addr 172.21.15.62:0/3814911117 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe3400640 1 --2- 172.21.15.62:0/3814911117 >> v2:172.21.15.81:3300/0 conn(0x7f2fe416b030 0x7f2fe416d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606515659 0 0) 0x7f2fe41628a0 con 0x7f2fe40b15c0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc8003520 con 0x7f2fe40b15c0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964412674 0 0) 0x7f2fc8003520 con 0x7f2fe40b15c0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 >> v2:172.21.15.81:3300/0 conn(0x7f2fe416b030 msgr2=0x7f2fe416d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 --2- 172.21.15.62:0/3814911117 >> v2:172.21.15.81:3300/0 conn(0x7f2fe416b030 0x7f2fe416d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 >> v1:172.21.15.81:6789/0 conn(0x7f2fe4070890 legacy=0x7f2fe40723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4163140 con 0x7f2fe40b15c0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f2fd4001670 con 0x7f2fe40b15c0 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.826+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 >> v1:172.21.15.62:6789/0 conn(0x7f2fe40b15c0 legacy=0x7f2fe40b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc80039b0 0x7f2fc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 0x7f2fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc80051c0 0x7f2fc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4162e60 con 0x7f2fc8004800 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc800c060 con 0x7f2fc80051c0 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc800c260 con 0x7f2fc80039b0 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 0x7f2fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2a00640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc80039b0 0x7f2fc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe3400640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc80051c0 0x7f2fc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 -- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc80039b0 msgr2=0x7f2fc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc80039b0 0x7f2fc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 -- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc80051c0 msgr2=0x7f2fc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fc80051c0 0x7f2fc800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe3400640 1 -- 172.21.15.62:0/3814911117 reap_dead start 2024-03-28T17:59:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.827+0000 7f2fe2000640 1 -- 172.21.15.62:0/3814911117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8003520 con 0x7f2fc8004800 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe2000640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 0x7f2fc8004c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0001750 tx=0x7f2fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3d3ae37ed7dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0001370 con 0x7f2fc8004800 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd00052d0 con 0x7f2fc8004800 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe1600640 1 -- 172.21.15.62:0/3814911117 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003830 con 0x7f2fc8004800 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe3e00640 1 -- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 msgr2=0x7f2fc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.829+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 0x7f2fc8004c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0001750 tx=0x7f2fc8005830 comp rx=0 tx=0).stop 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.830+0000 7f2fe3e00640 1 -- 172.21.15.62:0/3814911117 shutdown_connections 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.830+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/3814911117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fc8004800 0x7f2fc8004c80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.830+0000 7f2fe3e00640 1 -- 172.21.15.62:0/3814911117 >> 172.21.15.62:0/3814911117 conn(0x7f2fe4066dd0 msgr2=0x7f2fe40671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.830+0000 7f2fe3e00640 1 -- 172.21.15.62:0/3814911117 shutdown_connections 2024-03-28T17:59:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.830+0000 7f2fe3e00640 1 -- 172.21.15.62:0/3814911117 wait complete. 2024-03-28T17:59:51.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.831+0000 7f2fe3e00640 1 Processor -- start 2024-03-28T17:59:51.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.832+0000 7f2fe3e00640 1 -- start start 2024-03-28T17:59:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.832+0000 7f2fe3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8004800 0x7f2fe41687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.832+0000 7f2fe3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 0x7f2fe4168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40af130 con 0x7f2fe41f8310 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40b2b50 con 0x7f2fe415b840 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40af2b0 con 0x7f2fc8004800 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33072/0 (socket says 172.21.15.62:33072) 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 -- 172.21.15.62:0/133851086 learned_addr learned my addr 172.21.15.62:0/133851086 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2a00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8004800 0x7f2fe41687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3400640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 0x7f2fe4168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 -- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8004800 msgr2=0x7f2fe41687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8004800 0x7f2fe41687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 -- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 msgr2=0x7f2fe4168ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 0x7f2fe4168ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe2000640 1 -- 172.21.15.62:0/133851086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe415e7b0 con 0x7f2fe41f8310 2024-03-28T17:59:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.833+0000 7f2fe3400640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 0x7f2fe4168ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fe2000640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0041020 con 0x7f2fe41f8310 2024-03-28T17:59:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0005480 con 0x7f2fe41f8310 2024-03-28T17:59:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0007a30 con 0x7f2fe41f8310 2024-03-28T17:59:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe415ea90 con 0x7f2fe41f8310 2024-03-28T17:59:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.834+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe415f050 con 0x7f2fe41f8310 2024-03-28T17:59:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.835+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe404f400 con 0x7f2fe41f8310 2024-03-28T17:59:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.836+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2fd0044020 con 0x7f2fe41f8310 2024-03-28T17:59:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.837+0000 7f2fdaa00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 0x7f2fc406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.837+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2fd00ac590 con 0x7f2fe41f8310 2024-03-28T17:59:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.838+0000 7f2fe2a00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 0x7f2fc406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.839+0000 7f2fe2a00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 0x7f2fc406e090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4001050 tx=0x7f2fd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:51.842+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2fd004b030 con 0x7f2fe41f8310 2024-03-28T17:59:52.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.044+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fe406df70 con 0x7f2fe41f8310 2024-03-28T17:59:52.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.045+0000 7f2fdaa00640 1 -- 172.21.15.62:0/133851086 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2fd007af70 con 0x7f2fe41f8310 2024-03-28T17:59:52.045 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 msgr2=0x7f2fc406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 0x7f2fc406e090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4001050 tx=0x7f2fd4009040 comp rx=0 tx=0).stop 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 msgr2=0x7f2fe415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd0001120 comp rx=0 tx=0).stop 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 shutdown_connections 2024-03-28T17:59:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2fc406bbd0 0x7f2fc406e090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe415b840 0x7f2fe4168ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe41f8310 0x7f2fe415b120 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 --2- 172.21.15.62:0/133851086 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2fc8004800 0x7f2fe41687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 >> 172.21.15.62:0/133851086 conn(0x7f2fe4066dd0 msgr2=0x7f2fe406c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 shutdown_connections 2024-03-28T17:59:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.048+0000 7f2fe3e00640 1 -- 172.21.15.62:0/133851086 wait complete. 2024-03-28T17:59:52.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.294+0000 7f2776600640 1 Processor -- start 2024-03-28T17:59:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.294+0000 7f2776600640 1 -- start start 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f2776600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27700cf590 0x7f27700b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f2776600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2770160110 con 0x7f2770152f90 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f2776600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27701603c0 con 0x7f277006ccb0 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f2776600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27701606a0 con 0x7f27700cf590 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f276f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2770152f90 0x7f277015fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45392/0 (socket says 172.21.15.62:45392) 2024-03-28T17:59:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.296+0000 7f276f400640 1 -- 172.21.15.62:0/306840205 learned_addr learned my addr 172.21.15.62:0/306840205 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276fe00640 1 --2- 172.21.15.62:0/306840205 >> v2:172.21.15.81:3300/0 conn(0x7f27700cf590 0x7f27700b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887670340 0 0) 0x7f2770160110 con 0x7f2770152f90 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2754003520 con 0x7f2770152f90 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3222243262 0 0) 0x7f2754003520 con 0x7f2770152f90 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276fe00640 1 -- 172.21.15.62:0/306840205 >> v1:172.21.15.81:6789/0 conn(0x7f277006ccb0 legacy=0x7f27700cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276fe00640 1 -- 172.21.15.62:0/306840205 >> v1:172.21.15.62:6789/0 conn(0x7f2770152f90 legacy=0x7f277015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.297+0000 7f276fe00640 1 -- 172.21.15.62:0/306840205 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700b3cb0 con 0x7f27700cf590 2024-03-28T17:59:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.299+0000 7f276fe00640 1 --2- 172.21.15.62:0/306840205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27700cf590 0x7f27700b44b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2768009d90 tx=0x7f276802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e5eca6d09099e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.299+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768039070 con 0x7f27700cf590 2024-03-28T17:59:52.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.299+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276802c940 con 0x7f27700cf590 2024-03-28T17:59:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.299+0000 7f276e000640 1 -- 172.21.15.62:0/306840205 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760001670 con 0x7f27700cf590 2024-03-28T17:59:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 -- 172.21.15.62:0/306840205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27700cf590 msgr2=0x7f27700b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 --2- 172.21.15.62:0/306840205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27700cf590 0x7f27700b44b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2768009d90 tx=0x7f276802be10 comp rx=0 tx=0).stop 2024-03-28T17:59:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 -- 172.21.15.62:0/306840205 shutdown_connections 2024-03-28T17:59:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 --2- 172.21.15.62:0/306840205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27700cf590 0x7f27700b44b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 -- 172.21.15.62:0/306840205 >> 172.21.15.62:0/306840205 conn(0x7f2770066fb0 msgr2=0x7f27700673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 -- 172.21.15.62:0/306840205 shutdown_connections 2024-03-28T17:59:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.300+0000 7f2776600640 1 -- 172.21.15.62:0/306840205 wait complete. 2024-03-28T17:59:52.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.302+0000 7f2776600640 1 Processor -- start 2024-03-28T17:59:52.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.302+0000 7f2776600640 1 -- start start 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 0x7f27700cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 0x7f2770071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277015c750 con 0x7f27700cf590 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277015c5d0 con 0x7f2770152f90 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f2776600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700b49f0 con 0x7f277006ccb0 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33088/0 (socket says 172.21.15.62:33088) 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 -- 172.21.15.62:0/3494675772 learned_addr learned my addr 172.21.15.62:0/3494675772 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:52.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276fe00640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 0x7f2770071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276f400640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 0x7f27700cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 -- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 msgr2=0x7f27700cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 0x7f27700cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 -- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 msgr2=0x7f2770071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 0x7f2770071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.303+0000 7f276ea00640 1 -- 172.21.15.62:0/3494675772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700b0600 con 0x7f27700cf590 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f276fe00640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 0x7f2770071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f276f400640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 0x7f27700cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f276ea00640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f275c00c480 tx=0x7f275c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c0052d0 con 0x7f27700cf590 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c005470 con 0x7f27700cf590 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27700b08e0 con 0x7f27700cf590 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c012870 con 0x7f27700cf590 2024-03-28T17:59:52.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.304+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27700b0ea0 con 0x7f27700cf590 2024-03-28T17:59:52.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.306+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27700680c0 con 0x7f27700cf590 2024-03-28T17:59:52.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.306+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f275c012a10 con 0x7f27700cf590 2024-03-28T17:59:52.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.307+0000 7f2767400640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 0x7f275006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.307+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f275c08a840 con 0x7f27700cf590 2024-03-28T17:59:52.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.308+0000 7f276f400640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 0x7f275006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.309+0000 7f276f400640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 0x7f275006e090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2760001eb0 tx=0x7f2760009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.312+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f275c059220 con 0x7f27700cf590 2024-03-28T17:59:52.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.534+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27700b3500 con 0x7f27700cf590 2024-03-28T17:59:52.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.535+0000 7f2767400640 1 -- 172.21.15.62:0/3494675772 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f275c058b20 con 0x7f27700cf590 2024-03-28T17:59:52.535 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:52.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 msgr2=0x7f275006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 0x7f275006e090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2760001eb0 tx=0x7f2760009040 comp rx=0 tx=0).stop 2024-03-28T17:59:52.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 msgr2=0x7f277014a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f275c00c480 tx=0x7f275c00c940 comp rx=0 tx=0).stop 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 shutdown_connections 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f275006bbd0 0x7f275006e090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2770152f90 0x7f2770071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27700cf590 0x7f277014a710 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 --2- 172.21.15.62:0/3494675772 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f277006ccb0 0x7f27700cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 >> 172.21.15.62:0/3494675772 conn(0x7f2770066fb0 msgr2=0x7f2770067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 shutdown_connections 2024-03-28T17:59:52.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.538+0000 7f2776600640 1 -- 172.21.15.62:0/3494675772 wait complete. 2024-03-28T17:59:52.558 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T17:59:52.558 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:52.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.821+0000 7f5d36a00640 1 Processor -- start 2024-03-28T17:59:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.821+0000 7f5d36a00640 1 -- start start 2024-03-28T17:59:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.823+0000 7f5d36a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d3006ccb0 0x7f5d300cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.823+0000 7f5d36a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5d300cf3c0 0x7f5d300b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.823+0000 7f5d36a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d300cf7d0 con 0x7f5d30152f90 2024-03-28T17:59:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.823+0000 7f5d36a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d30160110 con 0x7f5d3006ccb0 2024-03-28T17:59:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.823+0000 7f5d36a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d301603c0 con 0x7f5d300cf3c0 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d3006ccb0 0x7f5d300cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d3006ccb0 0x7f5d300cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33094/0 (socket says 172.21.15.62:33094) 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 -- 172.21.15.62:0/2056739296 learned_addr learned my addr 172.21.15.62:0/2056739296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d34a00640 1 --2- 172.21.15.62:0/2056739296 >> v2:172.21.15.81:3300/0 conn(0x7f5d300cf3c0 0x7f5d300b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 -- 172.21.15.62:0/2056739296 >> v2:172.21.15.81:3300/0 conn(0x7f5d300cf3c0 msgr2=0x7f5d300b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 --2- 172.21.15.62:0/2056739296 >> v2:172.21.15.81:3300/0 conn(0x7f5d300cf3c0 0x7f5d300b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 -- 172.21.15.62:0/2056739296 >> v1:172.21.15.62:6789/0 conn(0x7f5d30152f90 legacy=0x7f5d3015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.824+0000 7f5d2f400640 1 -- 172.21.15.62:0/2056739296 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d301606a0 con 0x7f5d3006ccb0 2024-03-28T17:59:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d2f400640 1 --2- 172.21.15.62:0/2056739296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3006ccb0 0x7f5d300cee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d3009e040 tx=0x7f5d1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c8f1aeb5f4700 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d2ea00640 1 -- 172.21.15.62:0/2056739296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c034070 con 0x7f5d3006ccb0 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d2ea00640 1 -- 172.21.15.62:0/2056739296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c030860 con 0x7f5d3006ccb0 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d2ea00640 1 -- 172.21.15.62:0/2056739296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c030c30 con 0x7f5d3006ccb0 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d36a00640 1 -- 172.21.15.62:0/2056739296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3006ccb0 msgr2=0x7f5d300cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.826+0000 7f5d36a00640 1 --2- 172.21.15.62:0/2056739296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3006ccb0 0x7f5d300cee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d3009e040 tx=0x7f5d1c02b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 -- 172.21.15.62:0/2056739296 shutdown_connections 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 --2- 172.21.15.62:0/2056739296 >> v2:172.21.15.81:3300/0 conn(0x7f5d300cf3c0 0x7f5d300b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 --2- 172.21.15.62:0/2056739296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3006ccb0 0x7f5d300cee80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 -- 172.21.15.62:0/2056739296 >> 172.21.15.62:0/2056739296 conn(0x7f5d30066fb0 msgr2=0x7f5d300673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 -- 172.21.15.62:0/2056739296 shutdown_connections 2024-03-28T17:59:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.827+0000 7f5d36a00640 1 -- 172.21.15.62:0/2056739296 wait complete. 2024-03-28T17:59:52.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.828+0000 7f5d36a00640 1 Processor -- start 2024-03-28T17:59:52.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.829+0000 7f5d36a00640 1 -- start start 2024-03-28T17:59:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.829+0000 7f5d36a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 0x7f5d3014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.829+0000 7f5d36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.829+0000 7f5d36a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d30152f90 0x7f5d30071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.829+0000 7f5d36a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3015c5d0 con 0x7f5d300cf3c0 2024-03-28T17:59:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d36a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d300b3f90 con 0x7f5d30152f90 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d36a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3015c750 con 0x7f5d3006ccb0 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33100/0 (socket says 172.21.15.62:33100) 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 -- 172.21.15.62:0/4014103483 learned_addr learned my addr 172.21.15.62:0/4014103483 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2fe00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 0x7f5d3014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 -- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 msgr2=0x7f5d3014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d34a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d30152f90 0x7f5d30071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 0x7f5d3014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 -- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d30152f90 msgr2=0x7f5d30071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d30152f90 0x7f5d30071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2f400640 1 -- 172.21.15.62:0/4014103483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300727b0 con 0x7f5d300cf3c0 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.830+0000 7f5d2fe00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 0x7f5d3014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d2f400640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c000c00 tx=0x7f5d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c034040 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c039d90 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d30072a90 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c043610 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.831+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d30071b60 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.832+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30074a50 con 0x7f5d300cf3c0 2024-03-28T17:59:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.833+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5d1c037070 con 0x7f5d300cf3c0 2024-03-28T17:59:52.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.834+0000 7f5d27e00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 0x7f5d1006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:52.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.834+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5d1c0acc50 con 0x7f5d300cf3c0 2024-03-28T17:59:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.835+0000 7f5d2fe00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 0x7f5d1006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:52.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.836+0000 7f5d2fe00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 0x7f5d1006e0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d200015e0 tx=0x7f5d20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:52.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:52.839+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5d30074a50 con 0x7f5d300cf3c0 2024-03-28T17:59:53.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.041+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d300b3500 con 0x7f5d300cf3c0 2024-03-28T17:59:53.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.044+0000 7f5d27e00640 1 -- 172.21.15.62:0/4014103483 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f5d1c03e070 con 0x7f5d300cf3c0 2024-03-28T17:59:53.045 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:53.045 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 msgr2=0x7f5d1006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 0x7f5d1006e0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d200015e0 tx=0x7f5d20009040 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 msgr2=0x7f5d30144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c000c00 tx=0x7f5d1c0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 shutdown_connections 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.047+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5d1006bc00 0x7f5d1006e0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d30152f90 0x7f5d30071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300cf3c0 0x7f5d30144520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 --2- 172.21.15.62:0/4014103483 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5d3006ccb0 0x7f5d3014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 >> 172.21.15.62:0/4014103483 conn(0x7f5d30066fb0 msgr2=0x7f5d300b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 shutdown_connections 2024-03-28T17:59:53.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.048+0000 7f5d36a00640 1 -- 172.21.15.62:0/4014103483 wait complete. 2024-03-28T17:59:53.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:53.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:53.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.281+0000 7ff9a2c00640 1 Processor -- start 2024-03-28T17:59:53.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.282+0000 7ff9a2c00640 1 -- start start 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff99c0d1960 0x7ff99c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff99c071c00 0x7ff99c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99c0d3790 con 0x7ff99c071c00 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99c0b71b0 con 0x7ff99c0d1960 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.283+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0b6ef0 con 0x7ff99c0b2f40 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff99c071c00 0x7ff99c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff99c071c00 0x7ff99c15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33110/0 (socket says 172.21.15.62:33110) 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 -- 172.21.15.62:0/3174211796 learned_addr learned my addr 172.21.15.62:0/3174211796 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:53.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff99be00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff993400640 1 --2- 172.21.15.62:0/3174211796 >> v2:172.21.15.81:3300/0 conn(0x7ff99c0d1960 0x7ff99c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 -- 172.21.15.62:0/3174211796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 msgr2=0x7ff99c0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 -- 172.21.15.62:0/3174211796 >> v2:172.21.15.81:3300/0 conn(0x7ff99c0d1960 msgr2=0x7ff99c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 --2- 172.21.15.62:0/3174211796 >> v2:172.21.15.81:3300/0 conn(0x7ff99c0d1960 0x7ff99c0716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.284+0000 7ff9a0c00640 1 -- 172.21.15.62:0/3174211796 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0d2ef0 con 0x7ff99c071c00 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.285+0000 7ff9a0c00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c15d9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff99c068dc0 tx=0x7ff99402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b821315e374e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.286+0000 7ff99b400640 1 -- 172.21.15.62:0/3174211796 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994035020 con 0x7ff99c071c00 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.286+0000 7ff99b400640 1 -- 172.21.15.62:0/3174211796 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99402d040 con 0x7ff99c071c00 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.286+0000 7ff99b400640 1 -- 172.21.15.62:0/3174211796 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994031a80 con 0x7ff99c071c00 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.286+0000 7ff9a2c00640 1 -- 172.21.15.62:0/3174211796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 msgr2=0x7ff99c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.286+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c15d9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff99c068dc0 tx=0x7ff99402b900 comp rx=0 tx=0).stop 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 -- 172.21.15.62:0/3174211796 shutdown_connections 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c15d9e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/3174211796 >> v2:172.21.15.81:3300/0 conn(0x7ff99c0d1960 0x7ff99c0716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/3174211796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 -- 172.21.15.62:0/3174211796 >> 172.21.15.62:0/3174211796 conn(0x7ff99c0d3d90 msgr2=0x7ff99c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:53.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 -- 172.21.15.62:0/3174211796 shutdown_connections 2024-03-28T17:59:53.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.287+0000 7ff9a2c00640 1 -- 172.21.15.62:0/3174211796 wait complete. 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.288+0000 7ff9a2c00640 1 Processor -- start 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.288+0000 7ff9a2c00640 1 -- start start 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff99c0d1960 0x7ff99c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0d4bc0 con 0x7ff99c071c00 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0d34b0 con 0x7ff99c0d1960 2024-03-28T17:59:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.289+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0b3320 con 0x7ff99c0b2f40 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33112/0 (socket says 172.21.15.62:33112) 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 -- 172.21.15.62:0/1068964218 learned_addr learned my addr 172.21.15.62:0/1068964218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff993400640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff9a0c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff99c0d1960 0x7ff99c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 -- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 msgr2=0x7ff99c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 -- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff99c0d1960 msgr2=0x7ff99c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff99c0d1960 0x7ff99c0c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff99be00640 1 -- 172.21.15.62:0/1068964218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0b1ac0 con 0x7ff99c071c00 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.290+0000 7ff993400640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff99be00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007e60 tx=0x7ff98c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c014620 con 0x7ff99c071c00 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c019070 con 0x7ff99c071c00 2024-03-28T17:59:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c014a00 con 0x7ff99c071c00 2024-03-28T17:59:53.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c0b1da0 con 0x7ff99c071c00 2024-03-28T17:59:53.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.291+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c0b2360 con 0x7ff99c071c00 2024-03-28T17:59:53.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.292+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c0680c0 con 0x7ff99c071c00 2024-03-28T17:59:53.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.293+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff98c014ba0 con 0x7ff99c071c00 2024-03-28T17:59:53.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.295+0000 7ff998c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 0x7ff97806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.295+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff98c08b2e0 con 0x7ff99c071c00 2024-03-28T17:59:53.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.295+0000 7ff993400640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 0x7ff97806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.296+0000 7ff993400640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 0x7ff97806e090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff984006fd0 tx=0x7ff984008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.301+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff98c059de0 con 0x7ff99c071c00 2024-03-28T17:59:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.502+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff99c0716c0 con 0x7ff99c071c00 2024-03-28T17:59:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.503+0000 7ff998c00640 1 -- 172.21.15.62:0/1068964218 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff98c0596e0 con 0x7ff99c071c00 2024-03-28T17:59:53.503 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 msgr2=0x7ff97806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 0x7ff97806e090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff984006fd0 tx=0x7ff984008040 comp rx=0 tx=0).stop 2024-03-28T17:59:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 msgr2=0x7ff99c0ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007e60 tx=0x7ff98c00e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 shutdown_connections 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff97806bbd0 0x7ff97806e090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff99c0d1960 0x7ff99c0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff99c0b2f40 0x7ff99c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 --2- 172.21.15.62:0/1068964218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff99c071c00 0x7ff99c0ae610 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 >> 172.21.15.62:0/1068964218 conn(0x7ff99c0d3d90 msgr2=0x7ff99c0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.506+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 shutdown_connections 2024-03-28T17:59:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.507+0000 7ff9a2c00640 1 -- 172.21.15.62:0/1068964218 wait complete. 2024-03-28T17:59:53.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:53.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.751+0000 7f788c400640 1 Processor -- start 2024-03-28T17:59:53.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.752+0000 7f788c400640 1 -- start start 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 0x7f78840b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78840b2aa0 0x7f788406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 0x7f78840ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788406b0a0 con 0x7f78840b2aa0 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788406b660 con 0x7f78840b2640 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f788c400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840b3090 con 0x7f78840ce4b0 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f7889000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78840b2aa0 0x7f788406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f7889000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78840b2aa0 0x7f788406bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33118/0 (socket says 172.21.15.62:33118) 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.753+0000 7f7889000640 1 -- 172.21.15.62:0/3701811381 learned_addr learned my addr 172.21.15.62:0/3701811381 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889a00640 1 --2- 172.21.15.62:0/3701811381 >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 0x7f78840b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f788a400640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 0x7f78840ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889000640 1 -- 172.21.15.62:0/3701811381 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 msgr2=0x7f78840ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889000640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 0x7f78840ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889000640 1 -- 172.21.15.62:0/3701811381 >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 msgr2=0x7f78840b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889000640 1 --2- 172.21.15.62:0/3701811381 >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 0x7f78840b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889000640 1 -- 172.21.15.62:0/3701811381 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840b3370 con 0x7f78840b2aa0 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f788a400640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 0x7f78840ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.754+0000 7f7889a00640 1 --2- 172.21.15.62:0/3701811381 >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 0x7f78840b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:53.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.755+0000 7f7889000640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f788406bc60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78840cf530 tx=0x7f787002b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f2943125f7b587 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.755+0000 7f7883e00640 1 -- 172.21.15.62:0/3701811381 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870031760 con 0x7f78840b2aa0 2024-03-28T17:59:53.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.755+0000 7f7883e00640 1 -- 172.21.15.62:0/3701811381 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870037070 con 0x7f78840b2aa0 2024-03-28T17:59:53.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.755+0000 7f7883e00640 1 -- 172.21.15.62:0/3701811381 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870031b20 con 0x7f78840b2aa0 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 -- 172.21.15.62:0/3701811381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 msgr2=0x7f788406bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f788406bc60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78840cf530 tx=0x7f787002b800 comp rx=0 tx=0).stop 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 -- 172.21.15.62:0/3701811381 shutdown_connections 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840ce4b0 0x7f78840ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 --2- 172.21.15.62:0/3701811381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f788406bc60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 --2- 172.21.15.62:0/3701811381 >> v2:172.21.15.81:3300/0 conn(0x7f78840b2640 0x7f78840b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 -- 172.21.15.62:0/3701811381 >> 172.21.15.62:0/3701811381 conn(0x7f78840b3800 msgr2=0x7f78840b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 -- 172.21.15.62:0/3701811381 shutdown_connections 2024-03-28T17:59:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.756+0000 7f788c400640 1 -- 172.21.15.62:0/3701811381 wait complete. 2024-03-28T17:59:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.758+0000 7f788c400640 1 Processor -- start 2024-03-28T17:59:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.758+0000 7f788c400640 1 -- start start 2024-03-28T17:59:53.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840b2640 0x7f788415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78840ce4b0 0x7f78840afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788406c1a0 con 0x7f78840b2aa0 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840d1090 con 0x7f78840ce4b0 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788c400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788406c320 con 0x7f78840b2640 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f7889000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f7889000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33128/0 (socket says 172.21.15.62:33128) 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f7889000640 1 -- 172.21.15.62:0/479585968 learned_addr learned my addr 172.21.15.62:0/479585968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f7889a00640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840b2640 0x7f788415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.759+0000 7f788a400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78840ce4b0 0x7f78840afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 -- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840b2640 msgr2=0x7f788415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840b2640 0x7f788415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 -- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78840ce4b0 msgr2=0x7f78840afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78840ce4b0 0x7f78840afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 -- 172.21.15.62:0/479585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840aea70 con 0x7f78840b2aa0 2024-03-28T17:59:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7889000640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7870000c00 tx=0x7f78700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787003b5f0 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787003b7b0 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840aed50 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.760+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787003bba0 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.761+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840af310 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.762+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840680c0 con 0x7f78840b2aa0 2024-03-28T17:59:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.763+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f78700453f0 con 0x7f78840b2aa0 2024-03-28T17:59:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.764+0000 7f7881600640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 0x7f786406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.764+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f78700aca40 con 0x7f78840b2aa0 2024-03-28T17:59:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.764+0000 7f7889a00640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 0x7f786406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.765+0000 7f7889a00640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 0x7f786406e090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7874007fc0 tx=0x7f7874007550 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.768+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7870040070 con 0x7f78840b2aa0 2024-03-28T17:59:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.971+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78840d0e70 con 0x7f78840b2aa0 2024-03-28T17:59:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.971+0000 7f7881600640 1 -- 172.21.15.62:0/479585968 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f787007adc0 con 0x7f78840b2aa0 2024-03-28T17:59:53.972 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 msgr2=0x7f786406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 0x7f786406e090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7874007fc0 tx=0x7f7874007550 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 msgr2=0x7f78841f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7870000c00 tx=0x7f78700046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 shutdown_connections 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f786406bbd0 0x7f786406e090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78840ce4b0 0x7f78840afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78840b2aa0 0x7f78841f4be0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 --2- 172.21.15.62:0/479585968 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f78840b2640 0x7f788415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 >> 172.21.15.62:0/479585968 conn(0x7f78840b3800 msgr2=0x7f788415e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 shutdown_connections 2024-03-28T17:59:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:53.974+0000 7f788c400640 1 -- 172.21.15.62:0/479585968 wait complete. 2024-03-28T17:59:53.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.214+0000 7f480a600640 1 Processor -- start 2024-03-28T17:59:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.215+0000 7f480a600640 1 -- start start 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f480415e030 0x7f4804161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4804072ad0 0x7f4804161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48040723f0 0x7f48040ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48041629b0 con 0x7f48040723f0 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040b25b0 con 0x7f480415e030 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.216+0000 7f480a600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040b37a0 con 0x7f4804072ad0 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.217+0000 7f4803e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48040723f0 0x7f48040ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.217+0000 7f4803e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48040723f0 0x7f48040ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48998/0 (socket says 172.21.15.62:48998) 2024-03-28T17:59:54.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.217+0000 7f4803e00640 1 -- 172.21.15.62:0/4031576739 learned_addr learned my addr 172.21.15.62:0/4031576739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.217+0000 7f4803400640 1 --2- 172.21.15.62:0/4031576739 >> v2:172.21.15.81:3300/0 conn(0x7f480415e030 0x7f4804161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.217+0000 7f47fbe00640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4804072ad0 0x7f4804161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.218+0000 7f4803e00640 1 -- 172.21.15.62:0/4031576739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4804072ad0 msgr2=0x7f4804161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.218+0000 7f4803e00640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4804072ad0 0x7f4804161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.218+0000 7f4803e00640 1 -- 172.21.15.62:0/4031576739 >> v2:172.21.15.81:3300/0 conn(0x7f480415e030 msgr2=0x7f4804161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:54.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.218+0000 7f4803e00640 1 --2- 172.21.15.62:0/4031576739 >> v2:172.21.15.81:3300/0 conn(0x7f480415e030 0x7f4804161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.218+0000 7f4803e00640 1 -- 172.21.15.62:0/4031576739 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040b3470 con 0x7f48040723f0 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.219+0000 7f4803e00640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48040723f0 0x7f48040ae290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48040af470 tx=0x7f47fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d411562bac1e9517 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.219+0000 7f4802a00640 1 -- 172.21.15.62:0/4031576739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc02ca70 con 0x7f48040723f0 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.219+0000 7f4802a00640 1 -- 172.21.15.62:0/4031576739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc02cc10 con 0x7f48040723f0 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.219+0000 7f4802a00640 1 -- 172.21.15.62:0/4031576739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0343f0 con 0x7f48040723f0 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 -- 172.21.15.62:0/4031576739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48040723f0 msgr2=0x7f48040ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48040723f0 0x7f48040ae290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48040af470 tx=0x7f47fc02bed0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 -- 172.21.15.62:0/4031576739 shutdown_connections 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48040723f0 0x7f48040ae290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 --2- 172.21.15.62:0/4031576739 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4804072ad0 0x7f4804161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 --2- 172.21.15.62:0/4031576739 >> v2:172.21.15.81:3300/0 conn(0x7f480415e030 0x7f4804161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 -- 172.21.15.62:0/4031576739 >> 172.21.15.62:0/4031576739 conn(0x7f48040b1650 msgr2=0x7f48040b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:54.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 -- 172.21.15.62:0/4031576739 shutdown_connections 2024-03-28T17:59:54.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.220+0000 7f480a600640 1 -- 172.21.15.62:0/4031576739 wait complete. 2024-03-28T17:59:54.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.222+0000 7f480a600640 1 Processor -- start 2024-03-28T17:59:54.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.222+0000 7f480a600640 1 -- start start 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48040723f0 0x7f4804152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480415e030 0x7f48041554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480406cf30 con 0x7f4804072ad0 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804162480 con 0x7f48040723f0 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f480a600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480406d0b0 con 0x7f480415e030 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49006/0 (socket says 172.21.15.62:49006) 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 -- 172.21.15.62:0/1201045557 learned_addr learned my addr 172.21.15.62:0/1201045557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f4803400640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48040723f0 0x7f4804152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 -- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480415e030 msgr2=0x7f48041554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480415e030 0x7f48041554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 -- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48040723f0 msgr2=0x7f4804152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48040723f0 0x7f4804152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.223+0000 7f47fbe00640 1 -- 172.21.15.62:0/1201045557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48041611d0 con 0x7f4804072ad0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f47fbe00640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c820 tx=0x7f47ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec015070 con 0x7f4804072ad0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec004220 con 0x7f4804072ad0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480415e5b0 con 0x7f4804072ad0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00ce10 con 0x7f4804072ad0 2024-03-28T17:59:54.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.224+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480415eb70 con 0x7f4804072ad0 2024-03-28T17:59:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.226+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480415ed70 con 0x7f4804072ad0 2024-03-28T17:59:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.226+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f47ec0077d0 con 0x7f4804072ad0 2024-03-28T17:59:54.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.228+0000 7f47fb400640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 0x7f47e006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.228+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f47ec08ad30 con 0x7f4804072ad0 2024-03-28T17:59:54.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.228+0000 7f4803400640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 0x7f47e006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.230+0000 7f4803400640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 0x7f47e006e090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.234+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f480415ed70 con 0x7f4804072ad0 2024-03-28T17:59:54.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.443+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48040680c0 con 0x7f4804072ad0 2024-03-28T17:59:54.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.444+0000 7f47fb400640 1 -- 172.21.15.62:0/1201045557 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47ec059700 con 0x7f4804072ad0 2024-03-28T17:59:54.444 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:54.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 msgr2=0x7f47e006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 0x7f47e006e090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4008040 comp rx=0 tx=0).stop 2024-03-28T17:59:54.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 msgr2=0x7f4804154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c820 tx=0x7f47ec00cce0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 shutdown_connections 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e006bbd0 0x7f47e006e090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480415e030 0x7f48041554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4804072ad0 0x7f4804154f80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 --2- 172.21.15.62:0/1201045557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48040723f0 0x7f4804152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 >> 172.21.15.62:0/1201045557 conn(0x7f48040b1650 msgr2=0x7f48040b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 shutdown_connections 2024-03-28T17:59:54.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.448+0000 7f480a600640 1 -- 172.21.15.62:0/1201045557 wait complete. 2024-03-28T17:59:54.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.690+0000 7fd450a00640 1 Processor -- start 2024-03-28T17:59:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.691+0000 7fd450a00640 1 -- start start 2024-03-28T17:59:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd450a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd44c15b9c0 0x7fd44c0717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd450a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 0x7fd44c072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd450a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c070a40 con 0x7fd44c0722b0 2024-03-28T17:59:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd450a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c071330 con 0x7fd44c15b9c0 2024-03-28T17:59:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd450a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0b3af0 con 0x7fd44c06d1b0 2024-03-28T17:59:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.692+0000 7fd44b400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd44c15b9c0 0x7fd44c0717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44be00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 0x7fd44c072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd44c15b9c0 0x7fd44c0717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34510/0 (socket says 172.21.15.62:34510) 2024-03-28T17:59:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 -- 172.21.15.62:0/2892618295 learned_addr learned my addr 172.21.15.62:0/2892618295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 -- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 msgr2=0x7fd44c072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 0x7fd44c072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 -- 172.21.15.62:0/2892618295 >> v1:172.21.15.81:6789/0 conn(0x7fd44c0722b0 legacy=0x7fd44c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44b400640 1 -- 172.21.15.62:0/2892618295 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0b3810 con 0x7fd44c15b9c0 2024-03-28T17:59:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.693+0000 7fd44be00640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 0x7fd44c072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T17:59:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.695+0000 7fd44b400640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c15b9c0 0x7fd44c0717c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c223c17a6a70c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.695+0000 7fd44a000640 1 -- 172.21.15.62:0/2892618295 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c034070 con 0x7fd44c15b9c0 2024-03-28T17:59:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.695+0000 7fd44a000640 1 -- 172.21.15.62:0/2892618295 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c030840 con 0x7fd44c15b9c0 2024-03-28T17:59:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.695+0000 7fd44a000640 1 -- 172.21.15.62:0/2892618295 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c030c10 con 0x7fd44c15b9c0 2024-03-28T17:59:54.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 -- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c15b9c0 msgr2=0x7fd44c0717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c15b9c0 0x7fd44c0717c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c02b980 comp rx=0 tx=0).stop 2024-03-28T17:59:54.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 -- 172.21.15.62:0/2892618295 shutdown_connections 2024-03-28T17:59:54.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c06d1b0 0x7fd44c072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 --2- 172.21.15.62:0/2892618295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c15b9c0 0x7fd44c0717c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 -- 172.21.15.62:0/2892618295 >> 172.21.15.62:0/2892618295 conn(0x7fd44c0b1730 msgr2=0x7fd44c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:54.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 -- 172.21.15.62:0/2892618295 shutdown_connections 2024-03-28T17:59:54.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.696+0000 7fd450a00640 1 -- 172.21.15.62:0/2892618295 wait complete. 2024-03-28T17:59:54.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.698+0000 7fd450a00640 1 Processor -- start 2024-03-28T17:59:54.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.698+0000 7fd450a00640 1 -- start start 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c0722b0 0x7fd44c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c15b9c0 0x7fd44c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15bda0 con 0x7fd44c06d1b0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c070fc0 con 0x7fd44c0722b0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd450a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c071d00 con 0x7fd44c15b9c0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd44b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd44b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49008/0 (socket says 172.21.15.62:49008) 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.699+0000 7fd44b400640 1 -- 172.21.15.62:0/482473970 learned_addr learned my addr 172.21.15.62:0/482473970 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44aa00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c0722b0 0x7fd44c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44be00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c15b9c0 0x7fd44c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 -- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c15b9c0 msgr2=0x7fd44c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c15b9c0 0x7fd44c15cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 -- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c0722b0 msgr2=0x7fd44c156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c0722b0 0x7fd44c156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 -- 172.21.15.62:0/482473970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0cc910 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.700+0000 7fd44b400640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd43c02b9c0 tx=0x7fd43c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.701+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c048070 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.701+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c030640 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.701+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c034040 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.701+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c0ccbf0 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.701+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0cd1b0 con 0x7fd44c06d1b0 2024-03-28T17:59:54.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.702+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c073490 con 0x7fd44c06d1b0 2024-03-28T17:59:54.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.703+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd43c037070 con 0x7fd44c06d1b0 2024-03-28T17:59:54.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.704+0000 7fd443400640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 0x7fd42c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:54.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.704+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd43c0ac840 con 0x7fd44c06d1b0 2024-03-28T17:59:54.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.704+0000 7fd44aa00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 0x7fd42c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:54.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.706+0000 7fd44aa00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 0x7fd42c06e090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0cdee0 tx=0x7fd438009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.708+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd43c07b180 con 0x7fd44c06d1b0 2024-03-28T17:59:54.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.927+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd44c04f4e0 con 0x7fd44c06d1b0 2024-03-28T17:59:54.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.928+0000 7fd443400640 1 -- 172.21.15.62:0/482473970 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd43c07aa80 con 0x7fd44c06d1b0 2024-03-28T17:59:54.928 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 msgr2=0x7fd42c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 0x7fd42c06e090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0cdee0 tx=0x7fd438009040 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 msgr2=0x7fd44c1484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd43c02b9c0 tx=0x7fd43c004730 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 shutdown_connections 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd42c06bbd0 0x7fd42c06e090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd44c15b9c0 0x7fd44c15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd44c0722b0 0x7fd44c156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 --2- 172.21.15.62:0/482473970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd44c06d1b0 0x7fd44c1484f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 >> 172.21.15.62:0/482473970 conn(0x7fd44c0b1730 msgr2=0x7fd44c15e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 shutdown_connections 2024-03-28T17:59:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:54.931+0000 7fd450a00640 1 -- 172.21.15.62:0/482473970 wait complete. 2024-03-28T17:59:54.949 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi062.front.sepia.ceph.com... 2024-03-28T17:59:54.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:54.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-28T17:59:55.043 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T17:59:55.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-03-28T17:59:55.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:55.043 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-28T17:59:55.097 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:55.097 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:55.152 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:55.153 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:55.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.430+0000 7f5ceb200640 1 Processor -- start 2024-03-28T17:59:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.430+0000 7f5ceb200640 1 -- start start 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.431+0000 7f5ceb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5cec15b9c0 0x7f5cec0717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ceb200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5cec0722b0 0x7f5cec072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ceb200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec070a40 con 0x7f5cec06caa0 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ceb200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec071330 con 0x7f5cec15b9c0 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ceb200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec0b3af0 con 0x7f5cec0722b0 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5cec15b9c0 0x7f5cec0717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5cec15b9c0 0x7f5cec0717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49020/0 (socket says 172.21.15.62:49020) 2024-03-28T17:59:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce9e00640 1 -- 172.21.15.62:0/785986112 learned_addr learned my addr 172.21.15.62:0/785986112 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce3e00640 1 --2- 172.21.15.62:0/785986112 >> v2:172.21.15.81:3300/0 conn(0x7f5cec0722b0 0x7f5cec072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce9e00640 1 -- 172.21.15.62:0/785986112 >> v2:172.21.15.81:3300/0 conn(0x7f5cec0722b0 msgr2=0x7f5cec072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.432+0000 7f5ce9e00640 1 --2- 172.21.15.62:0/785986112 >> v2:172.21.15.81:3300/0 conn(0x7f5cec0722b0 0x7f5cec072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.433+0000 7f5ce9e00640 1 -- 172.21.15.62:0/785986112 >> v1:172.21.15.81:6789/0 conn(0x7f5cec06caa0 legacy=0x7f5cec06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.433+0000 7f5ce9e00640 1 -- 172.21.15.62:0/785986112 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0b3810 con 0x7f5cec15b9c0 2024-03-28T17:59:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.434+0000 7f5ce9e00640 1 --2- 172.21.15.62:0/785986112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec15b9c0 0x7f5cec0717c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0af670 tx=0x7f5cdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e54a374741d341f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.434+0000 7f5ce9400640 1 -- 172.21.15.62:0/785986112 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc035020 con 0x7f5cec15b9c0 2024-03-28T17:59:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.434+0000 7f5ce9400640 1 -- 172.21.15.62:0/785986112 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc031870 con 0x7f5cec15b9c0 2024-03-28T17:59:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.434+0000 7f5ce9400640 1 -- 172.21.15.62:0/785986112 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc031c40 con 0x7f5cec15b9c0 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 -- 172.21.15.62:0/785986112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec15b9c0 msgr2=0x7f5cec0717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 --2- 172.21.15.62:0/785986112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec15b9c0 0x7f5cec0717c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0af670 tx=0x7f5cdc02b910 comp rx=0 tx=0).stop 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 -- 172.21.15.62:0/785986112 shutdown_connections 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 --2- 172.21.15.62:0/785986112 >> v2:172.21.15.81:3300/0 conn(0x7f5cec0722b0 0x7f5cec072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 --2- 172.21.15.62:0/785986112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec15b9c0 0x7f5cec0717c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 -- 172.21.15.62:0/785986112 >> 172.21.15.62:0/785986112 conn(0x7f5cec0b1730 msgr2=0x7f5cec0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 -- 172.21.15.62:0/785986112 shutdown_connections 2024-03-28T17:59:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.435+0000 7f5ceb200640 1 -- 172.21.15.62:0/785986112 wait complete. 2024-03-28T17:59:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.437+0000 7f5ceb200640 1 Processor -- start 2024-03-28T17:59:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.437+0000 7f5ceb200640 1 -- start start 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5cec0722b0 0x7f5cec1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 0x7f5cec15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15bda0 con 0x7f5cec06caa0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec070fc0 con 0x7f5cec15b9c0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ceb200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec071d00 con 0x7f5cec0722b0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ce9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ce9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49032/0 (socket says 172.21.15.62:49032) 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ce9e00640 1 -- 172.21.15.62:0/3908649942 learned_addr learned my addr 172.21.15.62:0/3908649942 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5cea800640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 0x7f5cec15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.438+0000 7f5ce3e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5cec0722b0 0x7f5cec1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 -- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5cec0722b0 msgr2=0x7f5cec1f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5cec0722b0 0x7f5cec1f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 -- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 msgr2=0x7f5cec15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 0x7f5cec15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 -- 172.21.15.62:0/3908649942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec157fb0 con 0x7f5cec06caa0 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5cea800640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 0x7f5cec15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce9e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc040070 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc030d00 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc035040 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec158290 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.439+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec15c8f0 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.441+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec06b370 con 0x7f5cec06caa0 2024-03-28T17:59:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.442+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5cdc04b060 con 0x7f5cec06caa0 2024-03-28T17:59:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.442+0000 7f5ce2000640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 0x7f5cc806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.443+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5cdc0ac900 con 0x7f5cec06caa0 2024-03-28T17:59:55.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.443+0000 7f5ce3e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 0x7f5cc806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.444+0000 7f5ce3e00640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 0x7f5cc806e090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cec148eb0 tx=0x7f5cd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.447+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5cdc07b360 con 0x7f5cec06caa0 2024-03-28T17:59:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.652+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cec073490 con 0x7f5cec06caa0 2024-03-28T17:59:55.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.654+0000 7f5ce2000640 1 -- 172.21.15.62:0/3908649942 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f5cdc07ac60 con 0x7f5cec06caa0 2024-03-28T17:59:55.655 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:55.655 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 msgr2=0x7f5cc806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 0x7f5cc806e090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cec148eb0 tx=0x7f5cd4008040 comp rx=0 tx=0).stop 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 msgr2=0x7f5cec0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0046c0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 shutdown_connections 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5cc806bbd0 0x7f5cc806e090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cec15b9c0 0x7f5cec15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5cec0722b0 0x7f5cec1f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 --2- 172.21.15.62:0/3908649942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cec06caa0 0x7f5cec0cef00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 >> 172.21.15.62:0/3908649942 conn(0x7f5cec0b1730 msgr2=0x7f5cec06d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 shutdown_connections 2024-03-28T17:59:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.657+0000 7f5ceb200640 1 -- 172.21.15.62:0/3908649942 wait complete. 2024-03-28T17:59:55.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:55.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:55.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.900+0000 7fdca4600640 1 Processor -- start 2024-03-28T17:59:55.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.901+0000 7fdca4600640 1 -- start start 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca4600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c072ad0 0x7fdc9c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca4600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c0723f0 0x7fdc9c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca4600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0b3470 con 0x7fdc9c15e030 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca4600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c1629b0 con 0x7fdc9c072ad0 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca4600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b25b0 con 0x7fdc9c0723f0 2024-03-28T17:59:55.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca1200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c072ad0 0x7fdc9c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca1200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c072ad0 0x7fdc9c070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49034/0 (socket says 172.21.15.62:49034) 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca1200640 1 -- 172.21.15.62:0/833151572 learned_addr learned my addr 172.21.15.62:0/833151572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.903+0000 7fdca2600640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c0723f0 0x7fdc9c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.904+0000 7fdca1200640 1 -- 172.21.15.62:0/833151572 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c0723f0 msgr2=0x7fdc9c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.904+0000 7fdca1200640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c0723f0 0x7fdc9c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.904+0000 7fdca1200640 1 -- 172.21.15.62:0/833151572 >> v1:172.21.15.81:6789/0 conn(0x7fdc9c15e030 legacy=0x7fdc9c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.904+0000 7fdca1200640 1 -- 172.21.15.62:0/833151572 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c1626d0 con 0x7fdc9c072ad0 2024-03-28T17:59:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.905+0000 7fdca1200640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c070b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009870 tx=0x7fdc8802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbeaa81565e4e84 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.905+0000 7fdc9be00640 1 -- 172.21.15.62:0/833151572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88034070 con 0x7fdc9c072ad0 2024-03-28T17:59:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.905+0000 7fdc9be00640 1 -- 172.21.15.62:0/833151572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88030840 con 0x7fdc9c072ad0 2024-03-28T17:59:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.905+0000 7fdc9be00640 1 -- 172.21.15.62:0/833151572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88030c10 con 0x7fdc9c072ad0 2024-03-28T17:59:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 -- 172.21.15.62:0/833151572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c072ad0 msgr2=0x7fdc9c070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c070b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009870 tx=0x7fdc8802b980 comp rx=0 tx=0).stop 2024-03-28T17:59:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 -- 172.21.15.62:0/833151572 shutdown_connections 2024-03-28T17:59:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c0723f0 0x7fdc9c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 --2- 172.21.15.62:0/833151572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c070b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 -- 172.21.15.62:0/833151572 >> 172.21.15.62:0/833151572 conn(0x7fdc9c0b1650 msgr2=0x7fdc9c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 -- 172.21.15.62:0/833151572 shutdown_connections 2024-03-28T17:59:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.906+0000 7fdca4600640 1 -- 172.21.15.62:0/833151572 wait complete. 2024-03-28T17:59:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.908+0000 7fdca4600640 1 Processor -- start 2024-03-28T17:59:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.908+0000 7fdca4600640 1 -- start start 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c14a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c15e030 0x7fdc9c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c162110 con 0x7fdc9c0723f0 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b37a0 con 0x7fdc9c072ad0 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.909+0000 7fdca4600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c162290 con 0x7fdc9c15e030 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49048/0 (socket says 172.21.15.62:49048) 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 -- 172.21.15.62:0/1046141905 learned_addr learned my addr 172.21.15.62:0/1046141905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1200640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca2600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c15e030 0x7fdc9c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 -- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c15e030 msgr2=0x7fdc9c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c15e030 0x7fdc9c0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 -- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc9c072ad0 msgr2=0x7fdc9c14a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c14a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:55.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.910+0000 7fdca1c00640 1 -- 172.21.15.62:0/1046141905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c155d80 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdca1c00640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c8a0 tx=0x7fdc9000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90003b70 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90003d30 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000aa90 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c15e540 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.911+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c15f560 con 0x7fdc9c0723f0 2024-03-28T17:59:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.912+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c15db90 con 0x7fdc9c0723f0 2024-03-28T17:59:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.913+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdc9000ac30 con 0x7fdc9c0723f0 2024-03-28T17:59:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.915+0000 7fdc99600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 0x7fdc7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.915+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdc9008ab40 con 0x7fdc9c0723f0 2024-03-28T17:59:55.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.915+0000 7fdca1200640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 0x7fdc7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:55.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.916+0000 7fdca1200640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 0x7fdc7c06e090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc88036000 tx=0x7fdc8803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:55.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:55.920+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdc9c15db90 con 0x7fdc9c0723f0 2024-03-28T17:59:56.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.121+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc9c0680c0 con 0x7fdc9c0723f0 2024-03-28T17:59:56.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.122+0000 7fdc99600640 1 -- 172.21.15.62:0/1046141905 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc90059510 con 0x7fdc9c0723f0 2024-03-28T17:59:56.122 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:56.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 msgr2=0x7fdc7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 0x7fdc7c06e090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc88036000 tx=0x7fdc8803b040 comp rx=0 tx=0).stop 2024-03-28T17:59:56.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 msgr2=0x7fdc9c15f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c8a0 tx=0x7fdc9000cd60 comp rx=0 tx=0).stop 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 shutdown_connections 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdc7c06bbd0 0x7fdc7c06e090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdc9c15e030 0x7fdc9c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc9c072ad0 0x7fdc9c14a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 --2- 172.21.15.62:0/1046141905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0723f0 0x7fdc9c15f020 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 >> 172.21.15.62:0/1046141905 conn(0x7fdc9c0b1650 msgr2=0x7fdc9c0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 shutdown_connections 2024-03-28T17:59:56.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.125+0000 7fdca4600640 1 -- 172.21.15.62:0/1046141905 wait complete. 2024-03-28T17:59:56.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.370+0000 7f187ee00640 1 Processor -- start 2024-03-28T17:59:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.370+0000 7f187ee00640 1 -- start start 2024-03-28T17:59:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.371+0000 7f187ee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18780cec50 0x7f18780ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.371+0000 7f187ee00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18780b3210 0x7f18780d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.371+0000 7f187ee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1878161bb0 con 0x7f187815e2c0 2024-03-28T17:59:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.371+0000 7f187ee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1878161e90 con 0x7f18780cec50 2024-03-28T17:59:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.371+0000 7f187ee00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1878162170 con 0x7f18780b3210 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18780cec50 0x7f18780ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18780cec50 0x7f18780ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49058/0 (socket says 172.21.15.62:49058) 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 -- 172.21.15.62:0/993639046 learned_addr learned my addr 172.21.15.62:0/993639046 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187d000640 1 --1- 172.21.15.62:0/993639046 >> v1:172.21.15.62:6789/0 conn(0x7f187815e2c0 0x7f18780cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33804/0 (socket says 172.21.15.62:33804) 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187e400640 1 --2- 172.21.15.62:0/993639046 >> v2:172.21.15.81:3300/0 conn(0x7f18780b3210 0x7f18780d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 -- 172.21.15.62:0/993639046 >> v2:172.21.15.81:3300/0 conn(0x7f18780b3210 msgr2=0x7f18780d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 --2- 172.21.15.62:0/993639046 >> v2:172.21.15.81:3300/0 conn(0x7f18780b3210 0x7f18780d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 -- 172.21.15.62:0/993639046 >> v1:172.21.15.62:6789/0 conn(0x7f187815e2c0 legacy=0x7f18780cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.372+0000 7f187da00640 1 -- 172.21.15.62:0/993639046 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878162450 con 0x7f18780cec50 2024-03-28T17:59:56.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.373+0000 7f187da00640 1 --2- 172.21.15.62:0/993639046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780cec50 0x7f18780ce570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f187808c690 tx=0x7f186c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a913285fc9283 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.374+0000 7f1877e00640 1 -- 172.21.15.62:0/993639046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c034070 con 0x7f18780cec50 2024-03-28T17:59:56.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.374+0000 7f1877e00640 1 -- 172.21.15.62:0/993639046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c030860 con 0x7f18780cec50 2024-03-28T17:59:56.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.374+0000 7f1877e00640 1 -- 172.21.15.62:0/993639046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c030c30 con 0x7f18780cec50 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.374+0000 7f187ee00640 1 -- 172.21.15.62:0/993639046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780cec50 msgr2=0x7f18780ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.374+0000 7f187ee00640 1 --2- 172.21.15.62:0/993639046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780cec50 0x7f18780ce570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f187808c690 tx=0x7f186c02b9a0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 -- 172.21.15.62:0/993639046 shutdown_connections 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 --2- 172.21.15.62:0/993639046 >> v2:172.21.15.81:3300/0 conn(0x7f18780b3210 0x7f18780d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 --2- 172.21.15.62:0/993639046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780cec50 0x7f18780ce570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 -- 172.21.15.62:0/993639046 >> 172.21.15.62:0/993639046 conn(0x7f1878066fb0 msgr2=0x7f18780673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 -- 172.21.15.62:0/993639046 shutdown_connections 2024-03-28T17:59:56.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.375+0000 7f187ee00640 1 -- 172.21.15.62:0/993639046 wait complete. 2024-03-28T17:59:56.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.377+0000 7f187ee00640 1 Processor -- start 2024-03-28T17:59:56.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.377+0000 7f187ee00640 1 -- start start 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18780b3210 0x7f18781612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18780cec50 0x7f18781ea250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780d24f0 con 0x7f18780b5b70 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1878162a10 con 0x7f18780b3210 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187ee00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780d2670 con 0x7f18780cec50 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49066/0 (socket says 172.21.15.62:49066) 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187e400640 1 -- 172.21.15.62:0/1975515167 learned_addr learned my addr 172.21.15.62:0/1975515167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.378+0000 7f187da00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18780b3210 0x7f18781612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187d000640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18780cec50 0x7f18781ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 -- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18780cec50 msgr2=0x7f18781ea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18780cec50 0x7f18781ea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 -- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18780b3210 msgr2=0x7f18781612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18780b3210 0x7f18781612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 -- 172.21.15.62:0/1975515167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187815d290 con 0x7f18780b5b70 2024-03-28T17:59:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187e400640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1870007d10 tx=0x7f187000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870014420 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1870019070 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187815f950 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870014800 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.379+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187815ff10 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.381+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187804f400 con 0x7f18780b5b70 2024-03-28T17:59:56.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.382+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f18700149a0 con 0x7f18780b5b70 2024-03-28T17:59:56.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.383+0000 7f1875600640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 0x7f185806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.383+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f187008b1f0 con 0x7f18780b5b70 2024-03-28T17:59:56.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.383+0000 7f187da00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 0x7f185806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.384+0000 7f187da00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 0x7f185806e090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f186c036000 tx=0x7f186c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.387+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1870059cf0 con 0x7f18780b5b70 2024-03-28T17:59:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.589+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18780b28b0 con 0x7f18780b5b70 2024-03-28T17:59:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.590+0000 7f1875600640 1 -- 172.21.15.62:0/1975515167 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18700595f0 con 0x7f18780b5b70 2024-03-28T17:59:56.590 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 msgr2=0x7f185806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 0x7f185806e090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f186c036000 tx=0x7f186c03b040 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 msgr2=0x7f18780b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1870007d10 tx=0x7f187000e4b0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 shutdown_connections 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f185806bbd0 0x7f185806e090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18780b5b70 0x7f18780b6000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f18780cec50 0x7f18781ea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 --2- 172.21.15.62:0/1975515167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18780b3210 0x7f18781612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.592+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 >> 172.21.15.62:0/1975515167 conn(0x7f1878066fb0 msgr2=0x7f18780d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.593+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 shutdown_connections 2024-03-28T17:59:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.593+0000 7f187ee00640 1 -- 172.21.15.62:0/1975515167 wait complete. 2024-03-28T17:59:56.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:56.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.840+0000 7fb1efc00640 1 Processor -- start 2024-03-28T17:59:56.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.840+0000 7fb1efc00640 1 -- start start 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1efc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1e80cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1efc00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1e8072ad0 0x7fb1e80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1efc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e815d1b0 con 0x7fb1e80ae720 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1efc00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e8070d50 con 0x7fb1e8072ad0 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1efc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8071030 con 0x7fb1e8072690 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1ed200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb1e80ae720 0x7fb1e80aeb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33816/0 (socket says 172.21.15.62:33816) 2024-03-28T17:59:56.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.842+0000 7fb1ed200640 1 -- 172.21.15.62:0/718684327 learned_addr learned my addr 172.21.15.62:0/718684327 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:56.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1e80cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1edc00640 1 --2- 172.21.15.62:0/718684327 >> v2:172.21.15.81:3300/0 conn(0x7fb1e8072ad0 0x7fb1e80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836080082 0 0) 0x7fb1e815d1b0 con 0x7fb1e80ae720 2024-03-28T17:59:56.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1cc003520 con 0x7fb1e80ae720 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 27164643 0 0) 0x7fb1cc003520 con 0x7fb1e80ae720 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 msgr2=0x7fb1e80cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1e80cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 >> v2:172.21.15.81:3300/0 conn(0x7fb1e8072ad0 msgr2=0x7fb1e80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 --2- 172.21.15.62:0/718684327 >> v2:172.21.15.81:3300/0 conn(0x7fb1e8072ad0 0x7fb1e80ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e8071310 con 0x7fb1e80ae720 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fb1d8001670 con 0x7fb1e80ae720 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 >> v1:172.21.15.62:6789/0 conn(0x7fb1e80ae720 legacy=0x7fb1e80aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc003f30 0x7fb1cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1cc004d40 0x7fb1cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8070d50 con 0x7fb1cc003f30 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8071030 con 0x7fb1cc004d40 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.843+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc00b9c0 con 0x7fb1e8072690 2024-03-28T17:59:56.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc003f30 0x7fb1cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1ed200640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1edc00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1cc004d40 0x7fb1cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 -- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 msgr2=0x7fb1cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1e8072690 0x7fb1cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 -- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1cc004d40 msgr2=0x7fb1cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1cc004d40 0x7fb1cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.844+0000 7fb1e7e00640 1 -- 172.21.15.62:0/718684327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc003520 con 0x7fb1cc003f30 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.845+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc003f30 0x7fb1cc004800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009ac0 tx=0x7fb1cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5f0fb74e873856 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4039070 con 0x7fb1cc003f30 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d402c6b0 con 0x7fb1cc003f30 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1e7400640 1 -- 172.21.15.62:0/718684327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4009690 con 0x7fb1cc003f30 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 -- 172.21.15.62:0/718684327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc003f30 msgr2=0x7fb1cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 --2- 172.21.15.62:0/718684327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc003f30 0x7fb1cc004800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009ac0 tx=0x7fb1cc005830 comp rx=0 tx=0).stop 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1edc00640 1 -- 172.21.15.62:0/718684327 reap_dead start 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 -- 172.21.15.62:0/718684327 shutdown_connections 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 -- 172.21.15.62:0/718684327 >> 172.21.15.62:0/718684327 conn(0x7fb1e80b1650 msgr2=0x7fb1e80b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 -- 172.21.15.62:0/718684327 shutdown_connections 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.846+0000 7fb1efc00640 1 -- 172.21.15.62:0/718684327 wait complete. 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.848+0000 7fb1efc00640 1 Processor -- start 2024-03-28T17:59:56.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.848+0000 7fb1efc00640 1 -- start start 2024-03-28T17:59:56.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1cc003f30 0x7fb1e80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1e81f8210 0x7fb1e80ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8070a40 con 0x7fb1e80ade10 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e80718d0 con 0x7fb1e81f8210 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.849+0000 7fb1efc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8153030 con 0x7fb1cc003f30 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49090/0 (socket says 172.21.15.62:49090) 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 -- 172.21.15.62:0/1987976488 learned_addr learned my addr 172.21.15.62:0/1987976488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1ed200640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1cc003f30 0x7fb1e80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1e7e00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1e81f8210 0x7fb1e80ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 -- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1cc003f30 msgr2=0x7fb1e80b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1cc003f30 0x7fb1e80b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 -- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1e81f8210 msgr2=0x7fb1e80ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1e81f8210 0x7fb1e80ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.850+0000 7fb1edc00640 1 -- 172.21.15.62:0/1987976488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e81f8690 con 0x7fb1e80ade10 2024-03-28T17:59:56.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1edc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80722d0 tx=0x7fb1e00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0011db0 con 0x7fb1e80ade10 2024-03-28T17:59:56.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e001b070 con 0x7fb1e80ade10 2024-03-28T17:59:56.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0003990 con 0x7fb1e80ade10 2024-03-28T17:59:56.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e80b3a70 con 0x7fb1e80ade10 2024-03-28T17:59:56.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.851+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e80b4030 con 0x7fb1e80ade10 2024-03-28T17:59:56.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.852+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e80680c0 con 0x7fb1e80ade10 2024-03-28T17:59:56.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.853+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb1e0005d10 con 0x7fb1e80ade10 2024-03-28T17:59:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.854+0000 7fb1e4c00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 0x7fb1c806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.854+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb1e008afd0 con 0x7fb1e80ade10 2024-03-28T17:59:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.855+0000 7fb1ed200640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 0x7fb1c806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:56.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.856+0000 7fb1ed200640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 0x7fb1c806e0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8001050 tx=0x7fb1d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:56.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:56.859+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fb1e0014350 con 0x7fb1e80ade10 2024-03-28T17:59:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.070+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1e80cc790 con 0x7fb1e80ade10 2024-03-28T17:59:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.071+0000 7fb1e4c00640 1 -- 172.21.15.62:0/1987976488 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1e00599a0 con 0x7fb1e80ade10 2024-03-28T17:59:57.071 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 msgr2=0x7fb1c806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 0x7fb1c806e0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8001050 tx=0x7fb1d8009040 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 msgr2=0x7fb1e8142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80722d0 tx=0x7fb1e00115a0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 shutdown_connections 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fb1c806bc00 0x7fb1c806e0c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e80ade10 0x7fb1e8142b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1e81f8210 0x7fb1e80ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 --2- 172.21.15.62:0/1987976488 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fb1cc003f30 0x7fb1e80b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.073+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 >> 172.21.15.62:0/1987976488 conn(0x7fb1e80b1650 msgr2=0x7fb1e8151360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.074+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 shutdown_connections 2024-03-28T17:59:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.074+0000 7fb1efc00640 1 -- 172.21.15.62:0/1987976488 wait complete. 2024-03-28T17:59:57.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:57.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.322+0000 7fde71e00640 1 Processor -- start 2024-03-28T17:59:57.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.322+0000 7fde71e00640 1 -- start start 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde71e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde6c15dd60 0x7fde6c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde71e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0b6e60 con 0x7fde6c0b99f0 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde71e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0b7100 con 0x7fde6c0b5e00 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde71e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c0b6c30 con 0x7fde6c15dd60 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde6be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fde6c0b99f0 0x7fde6c0b56f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33822/0 (socket says 172.21.15.62:33822) 2024-03-28T17:59:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.323+0000 7fde6be00640 1 -- 172.21.15.62:0/2794431124 learned_addr learned my addr 172.21.15.62:0/2794431124 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde70a00640 1 --2- 172.21.15.62:0/2794431124 >> v2:172.21.15.81:3300/0 conn(0x7fde6c15dd60 0x7fde6c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120971277 0 0) 0x7fde6c0b6e60 con 0x7fde6c0b99f0 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde50003520 con 0x7fde6c0b99f0 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178485986 0 0) 0x7fde50003520 con 0x7fde6c0b99f0 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 >> v2:172.21.15.81:3300/0 conn(0x7fde6c15dd60 msgr2=0x7fde6c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 --2- 172.21.15.62:0/2794431124 >> v2:172.21.15.81:3300/0 conn(0x7fde6c15dd60 0x7fde6c0b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 >> v1:172.21.15.81:6789/0 conn(0x7fde6c0b5e00 legacy=0x7fde6c0b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0b73e0 con 0x7fde6c0b99f0 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fde58001690 con 0x7fde6c0b99f0 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 >> v1:172.21.15.62:6789/0 conn(0x7fde6c0b99f0 legacy=0x7fde6c0b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde500039b0 0x7fde50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 0x7fde50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde500051c0 0x7fde5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5000c040 con 0x7fde50004800 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5000c1f0 con 0x7fde500051c0 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.324+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5000c3f0 con 0x7fde500039b0 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 0x7fde50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 -- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde500039b0 msgr2=0x7fde50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde70a00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde500039b0 0x7fde50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde71400640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde500051c0 0x7fde5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde500039b0 0x7fde50003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 -- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde500051c0 msgr2=0x7fde5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde500051c0 0x7fde5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde6be00640 1 -- 172.21.15.62:0/2794431124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50003520 con 0x7fde50004800 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde71400640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde500051c0 0x7fde5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.325+0000 7fde71400640 1 -- 172.21.15.62:0/2794431124 reap_dead start 2024-03-28T17:59:57.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.326+0000 7fde6be00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 0x7fde50004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c210 tx=0x7fde50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5c40177d77e131 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58007d10 con 0x7fde50004800 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58030070 con 0x7fde50004800 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde6b400640 1 -- 172.21.15.62:0/2794431124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58005d70 con 0x7fde50004800 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 -- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 msgr2=0x7fde50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 0x7fde50004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c210 tx=0x7fde50005830 comp rx=0 tx=0).stop 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 -- 172.21.15.62:0/2794431124 shutdown_connections 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 --2- 172.21.15.62:0/2794431124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde50004800 0x7fde50004c80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 -- 172.21.15.62:0/2794431124 >> 172.21.15.62:0/2794431124 conn(0x7fde6c071ae0 msgr2=0x7fde6c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 -- 172.21.15.62:0/2794431124 shutdown_connections 2024-03-28T17:59:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.327+0000 7fde71e00640 1 -- 172.21.15.62:0/2794431124 wait complete. 2024-03-28T17:59:57.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.329+0000 7fde71e00640 1 Processor -- start 2024-03-28T17:59:57.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.329+0000 7fde71e00640 1 -- start start 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde50004800 0x7fde6c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 0x7fde6c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15e140 con 0x7fde6c0cbc00 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0b79a0 con 0x7fde6c15e2c0 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.330+0000 7fde71e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0b6370 con 0x7fde50004800 2024-03-28T17:59:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49106/0 (socket says 172.21.15.62:49106) 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 -- 172.21.15.62:0/3248308663 learned_addr learned my addr 172.21.15.62:0/3248308663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde70a00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde50004800 0x7fde6c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde6be00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 0x7fde6c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 -- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde50004800 msgr2=0x7fde6c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde50004800 0x7fde6c0cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 -- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 msgr2=0x7fde6c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 0x7fde6c0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 -- 172.21.15.62:0/3248308663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0cc160 con 0x7fde6c0cbc00 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde6be00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 0x7fde6c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:57.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.331+0000 7fde71400640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0d17b0 tx=0x7fde6400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.332+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64018070 con 0x7fde6c0cbc00 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.332+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64003da0 con 0x7fde6c0cbc00 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.332+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640052b0 con 0x7fde6c0cbc00 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.332+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c0c8b20 con 0x7fde6c0cbc00 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.332+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c0c90e0 con 0x7fde6c0cbc00 2024-03-28T17:59:57.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.333+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c0c8940 con 0x7fde6c0cbc00 2024-03-28T17:59:57.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.334+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fde6401b070 con 0x7fde6c0cbc00 2024-03-28T17:59:57.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.335+0000 7fde68c00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 0x7fde4c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.335+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fde6408a1c0 con 0x7fde6c0cbc00 2024-03-28T17:59:57.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.335+0000 7fde70a00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 0x7fde4c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.336+0000 7fde70a00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 0x7fde4c06e090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009a90 tx=0x7fde5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.339+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fde6c0c8940 con 0x7fde6c0cbc00 2024-03-28T17:59:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.555+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde6c072b90 con 0x7fde6c0cbc00 2024-03-28T17:59:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.556+0000 7fde68c00640 1 -- 172.21.15.62:0/3248308663 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde64058b90 con 0x7fde6c0cbc00 2024-03-28T17:59:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.558+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 msgr2=0x7fde4c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.558+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 0x7fde4c06e090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009a90 tx=0x7fde5c008040 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.558+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 msgr2=0x7fde6c0d3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.558+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0d17b0 tx=0x7fde6400d9d0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 shutdown_connections 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fde4c06bbd0 0x7fde4c06e090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde6c0cbc00 0x7fde6c0d3500 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c15e2c0 0x7fde6c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 --2- 172.21.15.62:0/3248308663 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fde50004800 0x7fde6c0cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 >> 172.21.15.62:0/3248308663 conn(0x7fde6c071ae0 msgr2=0x7fde6c0ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 shutdown_connections 2024-03-28T17:59:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.559+0000 7fde71e00640 1 -- 172.21.15.62:0/3248308663 wait complete. 2024-03-28T17:59:57.577 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-28T17:59:57.577 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T17:59:57.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T17:59:57.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.864+0000 7fa0e7200640 1 Processor -- start 2024-03-28T17:59:57.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.865+0000 7fa0e7200640 1 -- start start 2024-03-28T17:59:57.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.866+0000 7fa0e7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa0e00d1150 0x7fa0e006c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.866+0000 7fa0e7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00cf410 con 0x7fa0e00cdc30 2024-03-28T17:59:57.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.866+0000 7fa0e7200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e006b1d0 con 0x7fa0e00ce5a0 2024-03-28T17:59:57.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.866+0000 7fa0e7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e006bde0 con 0x7fa0e00d1150 2024-03-28T17:59:57.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0dfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa0e00cdc30 0x7fa0e00d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33828/0 (socket says 172.21.15.62:33828) 2024-03-28T17:59:57.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0dfe00640 1 -- 172.21.15.62:0/2103974865 learned_addr learned my addr 172.21.15.62:0/2103974865 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T17:59:57.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0df400640 1 --2- 172.21.15.62:0/2103974865 >> v2:172.21.15.62:3300/0 conn(0x7fa0e00d1150 0x7fa0e006c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0dea00640 1 -- 172.21.15.62:0/2103974865 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569777325 0 0) 0x7fa0e00cf410 con 0x7fa0e00cdc30 2024-03-28T17:59:57.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0dea00640 1 -- 172.21.15.62:0/2103974865 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c4003520 con 0x7fa0e00cdc30 2024-03-28T17:59:57.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0df400640 1 -- 172.21.15.62:0/2103974865 >> v1:172.21.15.81:6789/0 conn(0x7fa0e00ce5a0 legacy=0x7fa0e015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0df400640 1 -- 172.21.15.62:0/2103974865 >> v1:172.21.15.62:6789/0 conn(0x7fa0e00cdc30 legacy=0x7fa0e00d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.867+0000 7fa0df400640 1 -- 172.21.15.62:0/2103974865 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e006b4b0 con 0x7fa0e00d1150 2024-03-28T17:59:57.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0df400640 1 --2- 172.21.15.62:0/2103974865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e006c270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80097a0 tx=0x7fa0c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8326721d15842574 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0dea00640 1 -- 172.21.15.62:0/2103974865 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80315a0 con 0x7fa0e00d1150 2024-03-28T17:59:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0dea00640 1 -- 172.21.15.62:0/2103974865 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8037070 con 0x7fa0e00d1150 2024-03-28T17:59:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0dea00640 1 -- 172.21.15.62:0/2103974865 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8031940 con 0x7fa0e00d1150 2024-03-28T17:59:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0e7200640 1 -- 172.21.15.62:0/2103974865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 msgr2=0x7fa0e006c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.869+0000 7fa0e7200640 1 --2- 172.21.15.62:0/2103974865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e006c270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80097a0 tx=0x7fa0c802b840 comp rx=0 tx=0).stop 2024-03-28T17:59:57.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.870+0000 7fa0e7200640 1 -- 172.21.15.62:0/2103974865 shutdown_connections 2024-03-28T17:59:57.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.870+0000 7fa0e7200640 1 --2- 172.21.15.62:0/2103974865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e006c270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.870+0000 7fa0e7200640 1 -- 172.21.15.62:0/2103974865 >> 172.21.15.62:0/2103974865 conn(0x7fa0e00b3870 msgr2=0x7fa0e00b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:57.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.870+0000 7fa0e7200640 1 -- 172.21.15.62:0/2103974865 shutdown_connections 2024-03-28T17:59:57.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.870+0000 7fa0e7200640 1 -- 172.21.15.62:0/2103974865 wait complete. 2024-03-28T17:59:57.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.871+0000 7fa0e7200640 1 Processor -- start 2024-03-28T17:59:57.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.871+0000 7fa0e7200640 1 -- start start 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0e00cdc30 0x7fa0e01602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 0x7fa0e00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00ce010 con 0x7fa0e00d1150 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e006ba70 con 0x7fa0e00cdc30 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.872+0000 7fa0e7200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00ce190 con 0x7fa0e00ce5a0 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49126/0 (socket says 172.21.15.62:49126) 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 -- 172.21.15.62:0/434915522 learned_addr learned my addr 172.21.15.62:0/434915522 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0df400640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 0x7fa0e00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0dfe00640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0e00cdc30 0x7fa0e01602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 -- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 msgr2=0x7fa0e00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 0x7fa0e00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 -- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0e00cdc30 msgr2=0x7fa0e01602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0e00cdc30 0x7fa0e01602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0e5200640 1 -- 172.21.15.62:0/434915522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00b6820 con 0x7fa0e00d1150 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.873+0000 7fa0df400640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 0x7fa0e00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0e5200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8001eb0 tx=0x7fa0d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8017070 con 0x7fa0e00d1150 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80052d0 con 0x7fa0e00d1150 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003810 con 0x7fa0e00d1150 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e00b6b00 con 0x7fa0e00d1150 2024-03-28T17:59:57.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.874+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e0160830 con 0x7fa0e00d1150 2024-03-28T17:59:57.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.876+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e00680c0 con 0x7fa0e00d1150 2024-03-28T17:59:57.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.877+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa0d801a070 con 0x7fa0e00d1150 2024-03-28T17:59:57.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.877+0000 7fa0d7e00640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 0x7fa0bc06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:57.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.878+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa0d8089df0 con 0x7fa0e00d1150 2024-03-28T17:59:57.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.878+0000 7fa0dfe00640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 0x7fa0bc06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:57.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.879+0000 7fa0dfe00640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 0x7fa0bc06e0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0001560 tx=0x7fa0d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:57.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:57.882+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa0d8058940 con 0x7fa0e00d1150 2024-03-28T17:59:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.092+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0e00b2080 con 0x7fa0e00d1150 2024-03-28T17:59:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.095+0000 7fa0d7e00640 1 -- 172.21.15.62:0/434915522 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7fa0d8058240 con 0x7fa0e00d1150 2024-03-28T17:59:58.095 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T17:59:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 msgr2=0x7fa0bc06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 0x7fa0bc06e0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0001560 tx=0x7fa0d0009040 comp rx=0 tx=0).stop 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 msgr2=0x7fa0e01f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8001eb0 tx=0x7fa0d800c890 comp rx=0 tx=0).stop 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 shutdown_connections 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa0bc06bc20 0x7fa0bc06e0e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0e00d1150 0x7fa0e01f7ad0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa0e00ce5a0 0x7fa0e00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 --2- 172.21.15.62:0/434915522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0e00cdc30 0x7fa0e01602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 >> 172.21.15.62:0/434915522 conn(0x7fa0e00b3870 msgr2=0x7fa0e00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 shutdown_connections 2024-03-28T17:59:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.098+0000 7fa0e7200640 1 -- 172.21.15.62:0/434915522 wait complete. 2024-03-28T17:59:58.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T17:59:58.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T17:59:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.341+0000 7f665e400640 1 Processor -- start 2024-03-28T17:59:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.341+0000 7f665e400640 1 -- start start 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f665e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f665815e210 0x7f66580d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f665e400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66580ce780 0x7f66580d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f665e400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580d2510 con 0x7f66580d0b80 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f665e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66580d27b0 con 0x7f665815e210 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f665e400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66580d2a90 con 0x7f66580ce780 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f6657400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f665815e210 0x7f66580d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f6657400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f665815e210 0x7f66580d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49128/0 (socket says 172.21.15.62:49128) 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f6657400640 1 -- 172.21.15.62:0/4242112682 learned_addr learned my addr 172.21.15.62:0/4242112682 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:58.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.343+0000 7f6656a00640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66580ce780 0x7f66580d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.344+0000 7f6657400640 1 -- 172.21.15.62:0/4242112682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66580ce780 msgr2=0x7f66580d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:58.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.344+0000 7f6657400640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66580ce780 0x7f66580d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.344+0000 7f6657400640 1 -- 172.21.15.62:0/4242112682 >> v1:172.21.15.81:6789/0 conn(0x7f66580d0b80 legacy=0x7f66580b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:58.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.344+0000 7f6657400640 1 -- 172.21.15.62:0/4242112682 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580d2d70 con 0x7f665815e210 2024-03-28T17:59:58.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.345+0000 7f6657400640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f665815e210 0x7f66580d1fd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6648009e50 tx=0x7f664802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58047c60bcf2f7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.345+0000 7f6656000640 1 -- 172.21.15.62:0/4242112682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664802ca70 con 0x7f665815e210 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.345+0000 7f6656000640 1 -- 172.21.15.62:0/4242112682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664802cc10 con 0x7f665815e210 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.345+0000 7f6656000640 1 -- 172.21.15.62:0/4242112682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480343f0 con 0x7f665815e210 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 -- 172.21.15.62:0/4242112682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f665815e210 msgr2=0x7f66580d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f665815e210 0x7f66580d1fd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6648009e50 tx=0x7f664802bed0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 -- 172.21.15.62:0/4242112682 shutdown_connections 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66580ce780 0x7f66580d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 --2- 172.21.15.62:0/4242112682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f665815e210 0x7f66580d1fd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 -- 172.21.15.62:0/4242112682 >> 172.21.15.62:0/4242112682 conn(0x7f6658066dd0 msgr2=0x7f66580671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:58.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 -- 172.21.15.62:0/4242112682 shutdown_connections 2024-03-28T17:59:58.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.346+0000 7f665e400640 1 -- 172.21.15.62:0/4242112682 wait complete. 2024-03-28T17:59:58.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.348+0000 7f665e400640 1 Processor -- start 2024-03-28T17:59:58.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.348+0000 7f665e400640 1 -- start start 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66580ce780 0x7f66580c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f665815e210 0x7f66580cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580d1010 con 0x7f66580d0b80 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580d3330 con 0x7f66580ce780 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f665e400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665815e5f0 con 0x7f665815e210 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f6656a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f6656a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49136/0 (socket says 172.21.15.62:49136) 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f6656a00640 1 -- 172.21.15.62:0/1676242796 learned_addr learned my addr 172.21.15.62:0/1676242796 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f6657e00640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f665815e210 0x7f66580cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.349+0000 7f6657400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66580ce780 0x7f66580c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 -- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f665815e210 msgr2=0x7f66580cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f665815e210 0x7f66580cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 -- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66580ce780 msgr2=0x7f66580c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66580ce780 0x7f66580c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 -- 172.21.15.62:0/1676242796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658158340 con 0x7f66580d0b80 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f6656a00640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f664400d830 tx=0x7f664400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400b6c0 con 0x7f66580d0b80 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644019070 con 0x7f66580d0b80 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400ba80 con 0x7f66580d0b80 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6658158620 con 0x7f66580d0b80 2024-03-28T17:59:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.350+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6658158be0 con 0x7f66580d0b80 2024-03-28T17:59:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.352+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66580680c0 con 0x7f66580d0b80 2024-03-28T17:59:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.353+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f664401e460 con 0x7f66580d0b80 2024-03-28T17:59:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.353+0000 7f664f400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 0x7f663806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.354+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f664408b1d0 con 0x7f66580d0b80 2024-03-28T17:59:58.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.354+0000 7f6657400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 0x7f663806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.355+0000 7f6657400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 0x7f663806e090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6648035000 tx=0x7f6648030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.358+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6644059c40 con 0x7f66580d0b80 2024-03-28T17:59:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.562+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6658160ee0 con 0x7f66580d0b80 2024-03-28T17:59:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.563+0000 7f664f400640 1 -- 172.21.15.62:0/1676242796 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6644059540 con 0x7f66580d0b80 2024-03-28T17:59:58.563 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 msgr2=0x7f663806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 0x7f663806e090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6648035000 tx=0x7f6648030000 comp rx=0 tx=0).stop 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 msgr2=0x7f66580cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f664400d830 tx=0x7f664400dcf0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 shutdown_connections 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f663806bbd0 0x7f663806e090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f665815e210 0x7f66580cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66580d0b80 0x7f66580cd5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 --2- 172.21.15.62:0/1676242796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66580ce780 0x7f66580c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 >> 172.21.15.62:0/1676242796 conn(0x7f6658066dd0 msgr2=0x7f66580b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 shutdown_connections 2024-03-28T17:59:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.566+0000 7f665e400640 1 -- 172.21.15.62:0/1676242796 wait complete. 2024-03-28T17:59:58.585 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T17:59:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.800+0000 7f755c600640 1 Processor -- start 2024-03-28T17:59:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.800+0000 7f755c600640 1 -- start start 2024-03-28T17:59:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.802+0000 7f755c600640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75540ae720 0x7f75540aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.802+0000 7f755c600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7554072b00 0x7f755415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.802+0000 7f755c600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755415d1b0 con 0x7f7554072690 2024-03-28T17:59:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.802+0000 7f755c600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75540af040 con 0x7f7554072b00 2024-03-28T17:59:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.802+0000 7f755c600640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7554070d50 con 0x7f75540ae720 2024-03-28T17:59:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.803+0000 7f755a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7554072b00 0x7f755415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.803+0000 7f7559c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75540ae720 0x7f75540aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.803+0000 7f755a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7554072b00 0x7f755415ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49152/0 (socket says 172.21.15.62:49152) 2024-03-28T17:59:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.803+0000 7f755a600640 1 -- 172.21.15.62:0/1630677184 learned_addr learned my addr 172.21.15.62:0/1630677184 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.804+0000 7f755a600640 1 -- 172.21.15.62:0/1630677184 >> v2:172.21.15.81:3300/0 conn(0x7f75540ae720 msgr2=0x7f75540aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.804+0000 7f755a600640 1 --2- 172.21.15.62:0/1630677184 >> v2:172.21.15.81:3300/0 conn(0x7f75540ae720 0x7f75540aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.804+0000 7f755a600640 1 -- 172.21.15.62:0/1630677184 >> v1:172.21.15.81:6789/0 conn(0x7f7554072690 legacy=0x7f755415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.804+0000 7f755a600640 1 -- 172.21.15.62:0/1630677184 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554071030 con 0x7f7554072b00 2024-03-28T17:59:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.805+0000 7f755a600640 1 --2- 172.21.15.62:0/1630677184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072b00 0x7f755415ff50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f755406c770 tx=0x7f754c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1095c5e86f91815f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.805+0000 7f7553e00640 1 -- 172.21.15.62:0/1630677184 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c02cb20 con 0x7f7554072b00 2024-03-28T17:59:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.805+0000 7f7553e00640 1 -- 172.21.15.62:0/1630677184 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c02ccc0 con 0x7f7554072b00 2024-03-28T17:59:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.805+0000 7f7553e00640 1 -- 172.21.15.62:0/1630677184 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c034610 con 0x7f7554072b00 2024-03-28T17:59:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 -- 172.21.15.62:0/1630677184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072b00 msgr2=0x7f755415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 --2- 172.21.15.62:0/1630677184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072b00 0x7f755415ff50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f755406c770 tx=0x7f754c02bf80 comp rx=0 tx=0).stop 2024-03-28T17:59:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 -- 172.21.15.62:0/1630677184 shutdown_connections 2024-03-28T17:59:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 --2- 172.21.15.62:0/1630677184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072b00 0x7f755415ff50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 --2- 172.21.15.62:0/1630677184 >> v2:172.21.15.81:3300/0 conn(0x7f75540ae720 0x7f75540aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 -- 172.21.15.62:0/1630677184 >> 172.21.15.62:0/1630677184 conn(0x7f75540b1650 msgr2=0x7f75540b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:58.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 -- 172.21.15.62:0/1630677184 shutdown_connections 2024-03-28T17:59:58.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.806+0000 7f755c600640 1 -- 172.21.15.62:0/1630677184 wait complete. 2024-03-28T17:59:58.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.808+0000 7f755c600640 1 Processor -- start 2024-03-28T17:59:58.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.808+0000 7f755c600640 1 -- start start 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7554072b00 0x7f7554153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f75540ae720 0x7f75540b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540718d0 con 0x7f7554072690 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540715f0 con 0x7f7554072b00 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f755c600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7554070a40 con 0x7f75540ae720 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f7559c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f7559c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49164/0 (socket says 172.21.15.62:49164) 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.809+0000 7f7559c00640 1 -- 172.21.15.62:0/2650278576 learned_addr learned my addr 172.21.15.62:0/2650278576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559200640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7554072b00 0x7f7554153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f755a600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f75540ae720 0x7f75540b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 -- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f75540ae720 msgr2=0x7f75540b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f75540ae720 0x7f75540b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 -- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7554072b00 msgr2=0x7f7554153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7554072b00 0x7f7554153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 -- 172.21.15.62:0/2650278576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75541f5140 con 0x7f7554072690 2024-03-28T17:59:58.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7559c00640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f754800e870 tx=0x7f754800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.810+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548017070 con 0x7f7554072690 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.811+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800c8e0 con 0x7f7554072690 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.811+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800ccd0 con 0x7f7554072690 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.811+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75541f5420 con 0x7f7554072690 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.811+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75541f59e0 con 0x7f7554072690 2024-03-28T17:59:58.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.812+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75540680c0 con 0x7f7554072690 2024-03-28T17:59:58.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.813+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f754801a070 con 0x7f7554072690 2024-03-28T17:59:58.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.814+0000 7f7551600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 0x7f753406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:58.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.814+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f754808ace0 con 0x7f7554072690 2024-03-28T17:59:58.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.814+0000 7f7559200640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 0x7f753406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:58.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.815+0000 7f7559200640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 0x7f753406e090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f755406eee0 tx=0x7f7540009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:58.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:58.818+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7548059740 con 0x7f7554072690 2024-03-28T17:59:59.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.018+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75540b2420 con 0x7f7554072690 2024-03-28T17:59:59.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.019+0000 7f7551600640 1 -- 172.21.15.62:0/2650278576 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7548059040 con 0x7f7554072690 2024-03-28T17:59:59.019 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T17:59:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 msgr2=0x7f753406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 0x7f753406e090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f755406eee0 tx=0x7f7540009040 comp rx=0 tx=0).stop 2024-03-28T17:59:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 msgr2=0x7f7554135ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f754800e870 tx=0x7f754800ed30 comp rx=0 tx=0).stop 2024-03-28T17:59:59.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 shutdown_connections 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f753406bbd0 0x7f753406e090 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f75540ae720 0x7f75540b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7554072b00 0x7f7554153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 --2- 172.21.15.62:0/2650278576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7554072690 0x7f7554135ea0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 >> 172.21.15.62:0/2650278576 conn(0x7f75540b1650 msgr2=0x7f755415f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 shutdown_connections 2024-03-28T17:59:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.022+0000 7f755c600640 1 -- 172.21.15.62:0/2650278576 wait complete. 2024-03-28T17:59:59.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T17:59:59.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.266+0000 7f6e1f200640 1 Processor -- start 2024-03-28T17:59:59.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.267+0000 7f6e1f200640 1 -- start start 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.268+0000 7f6e1f200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 0x7f6e200b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.268+0000 7f6e1f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e20160190 0x7f6e2015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e200b44f0 0x7f6e200b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e200b4e80 con 0x7f6e20160190 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1f200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2015cfd0 con 0x7f6e200aee00 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015bec0 con 0x7f6e200b44f0 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e20160190 0x7f6e2015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e20160190 0x7f6e2015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49172/0 (socket says 172.21.15.62:49172) 2024-03-28T17:59:59.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 -- 172.21.15.62:0/1495156629 learned_addr learned my addr 172.21.15.62:0/1495156629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1de00640 1 --2- 172.21.15.62:0/1495156629 >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 0x7f6e200b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 -- 172.21.15.62:0/1495156629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e200b44f0 msgr2=0x7f6e200b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 --2- 172.21.15.62:0/1495156629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e200b44f0 0x7f6e200b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 -- 172.21.15.62:0/1495156629 >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 msgr2=0x7f6e200b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 --2- 172.21.15.62:0/1495156629 >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 0x7f6e200b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e17e00640 1 -- 172.21.15.62:0/1495156629 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200b1bf0 con 0x7f6e20160190 2024-03-28T17:59:59.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.269+0000 7f6e1de00640 1 --2- 172.21.15.62:0/1495156629 >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 0x7f6e200b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:59.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.271+0000 7f6e17e00640 1 --2- 172.21.15.62:0/1495156629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20160190 0x7f6e2015ca90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6e2006e9a0 tx=0x7f6e0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b89242625bf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.271+0000 7f6e1d400640 1 -- 172.21.15.62:0/1495156629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08035020 con 0x7f6e20160190 2024-03-28T17:59:59.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.271+0000 7f6e1d400640 1 -- 172.21.15.62:0/1495156629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08031870 con 0x7f6e20160190 2024-03-28T17:59:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.271+0000 7f6e1d400640 1 -- 172.21.15.62:0/1495156629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08031c40 con 0x7f6e20160190 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 -- 172.21.15.62:0/1495156629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20160190 msgr2=0x7f6e2015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 --2- 172.21.15.62:0/1495156629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20160190 0x7f6e2015ca90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6e2006e9a0 tx=0x7f6e0802b910 comp rx=0 tx=0).stop 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 -- 172.21.15.62:0/1495156629 shutdown_connections 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 --2- 172.21.15.62:0/1495156629 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e200b44f0 0x7f6e200b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 --2- 172.21.15.62:0/1495156629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20160190 0x7f6e2015ca90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 --2- 172.21.15.62:0/1495156629 >> v2:172.21.15.81:3300/0 conn(0x7f6e200aee00 0x7f6e200b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 -- 172.21.15.62:0/1495156629 >> 172.21.15.62:0/1495156629 conn(0x7f6e20066dd0 msgr2=0x7f6e200671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 -- 172.21.15.62:0/1495156629 shutdown_connections 2024-03-28T17:59:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.272+0000 7f6e1f200640 1 -- 172.21.15.62:0/1495156629 wait complete. 2024-03-28T17:59:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.273+0000 7f6e1f200640 1 Processor -- start 2024-03-28T17:59:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.274+0000 7f6e1f200640 1 -- start start 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e200b44f0 0x7f6e20157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 0x7f6e20156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015f990 con 0x7f6e200aee00 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015f6b0 con 0x7f6e200b44f0 2024-03-28T17:59:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015fc70 con 0x7f6e20160190 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49178/0 (socket says 172.21.15.62:49178) 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 -- 172.21.15.62:0/2406832660 learned_addr learned my addr 172.21.15.62:0/2406832660 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e17e00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e200b44f0 0x7f6e20157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1e800640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 0x7f6e20156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 -- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 msgr2=0x7f6e20156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 0x7f6e20156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 -- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e200b44f0 msgr2=0x7f6e20157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e200b44f0 0x7f6e20157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1de00640 1 -- 172.21.15.62:0/2406832660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e20157df0 con 0x7f6e200aee00 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.275+0000 7f6e1e800640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 0x7f6e20156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e1de00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000c7f0 tx=0x7f6e1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10015070 con 0x7f6e200aee00 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e10007a30 con 0x7f6e200aee00 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10007e00 con 0x7f6e200aee00 2024-03-28T17:59:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e201580d0 con 0x7f6e200aee00 2024-03-28T17:59:59.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.276+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e20158690 con 0x7f6e200aee00 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.277+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2004f400 con 0x7f6e200aee00 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.278+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6e10018070 con 0x7f6e200aee00 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.279+0000 7f6e16000640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 0x7f6dfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.279+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6e1008aac0 con 0x7f6e200aee00 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.280+0000 7f6e17e00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 0x7f6dfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.281+0000 7f6e17e00640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 0x7f6dfc06e090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e08036000 tx=0x7f6e0803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.284+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6e10059530 con 0x7f6e200aee00 2024-03-28T17:59:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.488+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e2015ba10 con 0x7f6e200aee00 2024-03-28T17:59:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.489+0000 7f6e16000640 1 -- 172.21.15.62:0/2406832660 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e10058e30 con 0x7f6e200aee00 2024-03-28T17:59:59.489 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 msgr2=0x7f6dfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 0x7f6dfc06e090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e08036000 tx=0x7f6e0803c040 comp rx=0 tx=0).stop 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 msgr2=0x7f6e20155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000c7f0 tx=0x7f6e1000cbf0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 shutdown_connections 2024-03-28T17:59:59.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6dfc06bbd0 0x7f6dfc06e090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6e20160190 0x7f6e20156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e200b44f0 0x7f6e20157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 --2- 172.21.15.62:0/2406832660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e200aee00 0x7f6e20155890 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 >> 172.21.15.62:0/2406832660 conn(0x7f6e20066dd0 msgr2=0x7f6e200b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 shutdown_connections 2024-03-28T17:59:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.492+0000 7f6e1f200640 1 -- 172.21.15.62:0/2406832660 wait complete. 2024-03-28T17:59:59.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T17:59:59.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.737+0000 7f9474a00640 1 Processor -- start 2024-03-28T17:59:59.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.738+0000 7f9474a00640 1 -- start start 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.739+0000 7f9474a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9470147530 0x7f9470162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.739+0000 7f9474a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f94700ae1e0 0x7f947006e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.739+0000 7f9474a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94700ae670 con 0x7f9470066dd0 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.739+0000 7f9474a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700ae910 con 0x7f9470147530 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.739+0000 7f9474a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700aebf0 con 0x7f94700ae1e0 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9470147530 0x7f9470162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9470147530 0x7f9470162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49182/0 (socket says 172.21.15.62:49182) 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 -- 172.21.15.62:0/4010947754 learned_addr learned my addr 172.21.15.62:0/4010947754 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 -- 172.21.15.62:0/4010947754 >> v2:172.21.15.81:3300/0 conn(0x7f94700ae1e0 msgr2=0x7f947006e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 --2- 172.21.15.62:0/4010947754 >> v2:172.21.15.81:3300/0 conn(0x7f94700ae1e0 0x7f947006e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 -- 172.21.15.62:0/4010947754 >> v1:172.21.15.81:6789/0 conn(0x7f9470066dd0 legacy=0x7f94700671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.740+0000 7f946e800640 1 -- 172.21.15.62:0/4010947754 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947006daa0 con 0x7f9470147530 2024-03-28T17:59:59.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.741+0000 7f946e800640 1 --2- 172.21.15.62:0/4010947754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f9470162430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f947006d550 tx=0x7f946002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd6ae1027239023 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.742+0000 7f946d400640 1 -- 172.21.15.62:0/4010947754 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946002ca80 con 0x7f9470147530 2024-03-28T17:59:59.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.742+0000 7f946d400640 1 -- 172.21.15.62:0/4010947754 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946002cc20 con 0x7f9470147530 2024-03-28T17:59:59.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.742+0000 7f946d400640 1 -- 172.21.15.62:0/4010947754 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600343f0 con 0x7f9470147530 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.742+0000 7f9474a00640 1 -- 172.21.15.62:0/4010947754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 msgr2=0x7f9470162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.742+0000 7f9474a00640 1 --2- 172.21.15.62:0/4010947754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f9470162430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f947006d550 tx=0x7f946002bee0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 -- 172.21.15.62:0/4010947754 shutdown_connections 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 --2- 172.21.15.62:0/4010947754 >> v2:172.21.15.81:3300/0 conn(0x7f94700ae1e0 0x7f947006e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 --2- 172.21.15.62:0/4010947754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f9470162430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 -- 172.21.15.62:0/4010947754 >> 172.21.15.62:0/4010947754 conn(0x7f94700b2800 msgr2=0x7f94700b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 -- 172.21.15.62:0/4010947754 shutdown_connections 2024-03-28T17:59:59.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.743+0000 7f9474a00640 1 -- 172.21.15.62:0/4010947754 wait complete. 2024-03-28T17:59:59.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.745+0000 7f9474a00640 1 Processor -- start 2024-03-28T17:59:59.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.745+0000 7f9474a00640 1 -- start start 2024-03-28T17:59:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470066dd0 0x7f94700b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 0x7f94700c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947006eac0 con 0x7f9470147530 2024-03-28T17:59:59.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947006e060 con 0x7f94700ae1e0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f9474a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9470147910 con 0x7f9470066dd0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f946f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f946f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49196/0 (socket says 172.21.15.62:49196) 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f946f200640 1 -- 172.21.15.62:0/3606435981 learned_addr learned my addr 172.21.15.62:0/3606435981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f946de00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 0x7f94700c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.746+0000 7f946e800640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470066dd0 0x7f94700b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 -- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470066dd0 msgr2=0x7f94700b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470066dd0 0x7f94700b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 -- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 msgr2=0x7f94700c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 0x7f94700c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 -- 172.21.15.62:0/3606435981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470133e50 con 0x7f9470147530 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946de00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 0x7f94700c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f946f200640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f946800c410 tx=0x7f946800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468011790 con 0x7f9470147530 2024-03-28T17:59:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468017070 con 0x7f9470147530 2024-03-28T17:59:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468011bb0 con 0x7f9470147530 2024-03-28T17:59:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9470134130 con 0x7f9470147530 2024-03-28T17:59:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.747+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94701346f0 con 0x7f9470147530 2024-03-28T17:59:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.749+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9470073340 con 0x7f9470147530 2024-03-28T17:59:59.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.749+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f946801c460 con 0x7f9470147530 2024-03-28T17:59:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.750+0000 7f9466a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 0x7f944c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:59.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.751+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f946808a790 con 0x7f9470147530 2024-03-28T17:59:59.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.751+0000 7f946e800640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 0x7f944c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:59.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.752+0000 7f946e800640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 0x7f944c06e090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9460035000 tx=0x7f9460030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.755+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9468021020 con 0x7f9470147530 2024-03-28T17:59:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.965+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f947004f4e0 con 0x7f9470147530 2024-03-28T17:59:59.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.966+0000 7f9466a00640 1 -- 172.21.15.62:0/3606435981 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9468059200 con 0x7f9470147530 2024-03-28T17:59:59.966 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 msgr2=0x7f944c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 0x7f944c06e090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9460035000 tx=0x7f9460030000 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 msgr2=0x7f94700b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f946800c410 tx=0x7f946800c8d0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 shutdown_connections 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f944c06bbd0 0x7f944c06e090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9470147530 0x7f94700b5d40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94700ae1e0 0x7f94700c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 --2- 172.21.15.62:0/3606435981 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9470066dd0 0x7f94700b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.968+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 >> 172.21.15.62:0/3606435981 conn(0x7f94700b2800 msgr2=0x7f94701429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.969+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 shutdown_connections 2024-03-28T17:59:59.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T17:59:59.969+0000 7f9474a00640 1 -- 172.21.15.62:0/3606435981 wait complete. 2024-03-28T17:59:59.987 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-03-28T17:59:59.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T17:59:59.988 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-28T18:00:00.085 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T18:00:00.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-28T18:00:00.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:00.085 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-28T18:00:00.130 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T18:00:00.131 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:00.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.352+0000 7f7472c00640 1 Processor -- start 2024-03-28T18:00:00.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.353+0000 7f7472c00640 1 -- start start 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.354+0000 7f7472c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.354+0000 7f7472c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f746c0cf9d0 0x7f746c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.354+0000 7f7472c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c06b240 con 0x7f746c0ceb60 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.354+0000 7f7472c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c06bb70 con 0x7f746c0cf9d0 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.354+0000 7f7472c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cef90 con 0x7f746c161930 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f7470c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f746c0ceb60 0x7f746c0cfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33842/0 (socket says 172.21.15.62:33842) 2024-03-28T18:00:00.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f7470c00640 1 -- 172.21.15.62:0/858342701 learned_addr learned my addr 172.21.15.62:0/858342701 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f746b400640 1 --2- 172.21.15.62:0/858342701 >> v2:172.21.15.62:3300/0 conn(0x7f746c0cf9d0 0x7f746c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f746be00640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f746aa00640 1 -- 172.21.15.62:0/858342701 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189215533 0 0) 0x7f746c06b240 con 0x7f746c0ceb60 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.355+0000 7f746aa00640 1 -- 172.21.15.62:0/858342701 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f744c003520 con 0x7f746c0ceb60 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.356+0000 7f746b400640 1 -- 172.21.15.62:0/858342701 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 msgr2=0x7f746c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.356+0000 7f746b400640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.356+0000 7f746b400640 1 -- 172.21.15.62:0/858342701 >> v1:172.21.15.62:6789/0 conn(0x7f746c0ceb60 legacy=0x7f746c0cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.356+0000 7f746b400640 1 -- 172.21.15.62:0/858342701 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c06b520 con 0x7f746c0cf9d0 2024-03-28T18:00:00.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.357+0000 7f746b400640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0cf9d0 0x7f746c0ce620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7454009d90 tx=0x7f745402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420d3797db45e2a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.357+0000 7f746aa00640 1 -- 172.21.15.62:0/858342701 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454039070 con 0x7f746c0cf9d0 2024-03-28T18:00:00.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.357+0000 7f746aa00640 1 -- 172.21.15.62:0/858342701 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745402cc50 con 0x7f746c0cf9d0 2024-03-28T18:00:00.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.357+0000 7f746aa00640 1 -- 172.21.15.62:0/858342701 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454034440 con 0x7f746c0cf9d0 2024-03-28T18:00:00.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 -- 172.21.15.62:0/858342701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0cf9d0 msgr2=0x7f746c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0cf9d0 0x7f746c0ce620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7454009d90 tx=0x7f745402bea0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 -- 172.21.15.62:0/858342701 shutdown_connections 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0cf9d0 0x7f746c0ce620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 --2- 172.21.15.62:0/858342701 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 -- 172.21.15.62:0/858342701 >> 172.21.15.62:0/858342701 conn(0x7f746c0b38e0 msgr2=0x7f746c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 -- 172.21.15.62:0/858342701 shutdown_connections 2024-03-28T18:00:00.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.358+0000 7f7472c00640 1 -- 172.21.15.62:0/858342701 wait complete. 2024-03-28T18:00:00.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.360+0000 7f7472c00640 1 Processor -- start 2024-03-28T18:00:00.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.360+0000 7f7472c00640 1 -- start start 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f746c0cf9d0 0x7f746c1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cdb30 con 0x7f746c0ceb60 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cd9b0 con 0x7f746c0cf9d0 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f7472c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c06d010 con 0x7f746c161930 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49208/0 (socket says 172.21.15.62:49208) 2024-03-28T18:00:00.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 -- 172.21.15.62:0/1356456824 learned_addr learned my addr 172.21.15.62:0/1356456824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:00.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 -- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 msgr2=0x7f746c06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 -- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f746c0cf9d0 msgr2=0x7f746c1500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f746c0cf9d0 0x7f746c1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.361+0000 7f746be00640 1 -- 172.21.15.62:0/1356456824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0c5d60 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f746be00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e870 tx=0x7f745c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c017070 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00c940 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00cd30 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c0c6040 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.362+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c0c6600 con 0x7f746c0ceb60 2024-03-28T18:00:00.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.363+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c0687d0 con 0x7f746c0ceb60 2024-03-28T18:00:00.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.364+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f745c01a070 con 0x7f746c0ceb60 2024-03-28T18:00:00.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.365+0000 7f7463e00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 0x7f744806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.365+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f745c08ad00 con 0x7f746c0ceb60 2024-03-28T18:00:00.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.366+0000 7f746b400640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 0x7f744806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.367+0000 7f746b400640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 0x7f744806e090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7454036000 tx=0x7f745402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.370+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f745c059800 con 0x7f746c0ceb60 2024-03-28T18:00:00.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.613+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f746c0d0910 con 0x7f746c0ceb60 2024-03-28T18:00:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.614+0000 7f7463e00640 1 -- 172.21.15.62:0/1356456824 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f745c059100 con 0x7f746c0ceb60 2024-03-28T18:00:00.614 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:00.615 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T18:00:00.615 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 msgr2=0x7f744806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 0x7f744806e090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7454036000 tx=0x7f745402e000 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 msgr2=0x7f746c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e870 tx=0x7f745c00ed30 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 shutdown_connections 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f744806bbd0 0x7f744806e090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f746c161930 0x7f746c06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f746c0cf9d0 0x7f746c1500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 --2- 172.21.15.62:0/1356456824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f746c0ceb60 0x7f746c0c8f40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 >> 172.21.15.62:0/1356456824 conn(0x7f746c0b38e0 msgr2=0x7f746c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 shutdown_connections 2024-03-28T18:00:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.617+0000 7f7472c00640 1 -- 172.21.15.62:0/1356456824 wait complete. 2024-03-28T18:00:00.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.858+0000 7ff073e00640 1 Processor -- start 2024-03-28T18:00:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.858+0000 7ff073e00640 1 -- start start 2024-03-28T18:00:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.860+0000 7ff073e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff07415dd60 0x7ff0740b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.860+0000 7ff073e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0740b5e00 0x7ff0740b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.860+0000 7ff073e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740b6e60 con 0x7ff0740b99f0 2024-03-28T18:00:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.860+0000 7ff073e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740b6c30 con 0x7ff0740b5e00 2024-03-28T18:00:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.860+0000 7ff073e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b7100 con 0x7ff07415dd60 2024-03-28T18:00:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0740b5e00 0x7ff0740b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0740b5e00 0x7ff0740b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34714/0 (socket says 172.21.15.62:34714) 2024-03-28T18:00:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff072a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff07415dd60 0x7ff0740b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 -- 172.21.15.62:0/1761445233 learned_addr learned my addr 172.21.15.62:0/1761445233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 -- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff07415dd60 msgr2=0x7ff0740b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 --2- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff07415dd60 0x7ff0740b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 -- 172.21.15.62:0/1761445233 >> v1:172.21.15.81:6789/0 conn(0x7ff0740b99f0 legacy=0x7ff0740b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.861+0000 7ff073400640 1 -- 172.21.15.62:0/1761445233 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b73e0 con 0x7ff0740b5e00 2024-03-28T18:00:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.863+0000 7ff073400640 1 --2- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740b5e00 0x7ff0740b7ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009870 tx=0x7ff06c02ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5dcc88c6c79b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.863+0000 7ff071600640 1 -- 172.21.15.62:0/1761445233 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c02f840 con 0x7ff0740b5e00 2024-03-28T18:00:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.863+0000 7ff071600640 1 -- 172.21.15.62:0/1761445233 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c02f9e0 con 0x7ff0740b5e00 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.863+0000 7ff071600640 1 -- 172.21.15.62:0/1761445233 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c02fdb0 con 0x7ff0740b5e00 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 -- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740b5e00 msgr2=0x7ff0740b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 --2- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740b5e00 0x7ff0740b7ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009870 tx=0x7ff06c02ba20 comp rx=0 tx=0).stop 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 -- 172.21.15.62:0/1761445233 shutdown_connections 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 --2- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740b5e00 0x7ff0740b7ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 --2- 172.21.15.62:0/1761445233 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff07415dd60 0x7ff0740b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 -- 172.21.15.62:0/1761445233 >> 172.21.15.62:0/1761445233 conn(0x7ff074071ae0 msgr2=0x7ff074071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 -- 172.21.15.62:0/1761445233 shutdown_connections 2024-03-28T18:00:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.864+0000 7ff073e00640 1 -- 172.21.15.62:0/1761445233 wait complete. 2024-03-28T18:00:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.866+0000 7ff073e00640 1 Processor -- start 2024-03-28T18:00:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.866+0000 7ff073e00640 1 -- start start 2024-03-28T18:00:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0740b5e00 0x7ff0740c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff07415dd60 0x7ff07415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07415e140 con 0x7ff0740b99f0 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b79a0 con 0x7ff07415dd60 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff073e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b6370 con 0x7ff0740b5e00 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff072000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff072000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49224/0 (socket says 172.21.15.62:49224) 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.867+0000 7ff072000640 1 -- 172.21.15.62:0/593217541 learned_addr learned my addr 172.21.15.62:0/593217541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072a00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0740b5e00 0x7ff0740c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff073400640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff07415dd60 0x7ff07415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 -- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0740b5e00 msgr2=0x7ff0740c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0740b5e00 0x7ff0740c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 -- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff07415dd60 msgr2=0x7ff07415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff07415dd60 0x7ff07415fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 -- 172.21.15.62:0/593217541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074159830 con 0x7ff0740b99f0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff072000640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c400 tx=0x7ff05c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0052d0 con 0x7ff0740b99f0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c005490 con 0x7ff0740b99f0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c007260 con 0x7ff0740b99f0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.868+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740c7d20 con 0x7ff0740b99f0 2024-03-28T18:00:00.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.869+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0740c82e0 con 0x7ff0740b99f0 2024-03-28T18:00:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.870+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff074072b90 con 0x7ff0740b99f0 2024-03-28T18:00:00.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.871+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff05c007400 con 0x7ff0740b99f0 2024-03-28T18:00:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.872+0000 7ff06aa00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 0x7ff05006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.872+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff05c08a070 con 0x7ff0740b99f0 2024-03-28T18:00:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.873+0000 7ff072a00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 0x7ff05006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.874+0000 7ff072a00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 0x7ff05006e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0640073f0 tx=0x7ff06400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:00.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:00.877+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7ff074159830 con 0x7ff0740b99f0 2024-03-28T18:00:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.081+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff07404f4e0 con 0x7ff0740b99f0 2024-03-28T18:00:01.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.084+0000 7ff06aa00640 1 -- 172.21.15.62:0/593217541 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7ff05c058ad0 con 0x7ff0740b99f0 2024-03-28T18:00:01.085 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:01.085 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 msgr2=0x7ff05006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 0x7ff05006e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0640073f0 tx=0x7ff06400b040 comp rx=0 tx=0).stop 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 msgr2=0x7ff07414d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c400 tx=0x7ff05c00c8c0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 shutdown_connections 2024-03-28T18:00:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7ff05006bbd0 0x7ff05006e090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff07415dd60 0x7ff07415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0740b99f0 0x7ff07414d130 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 --2- 172.21.15.62:0/593217541 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7ff0740b5e00 0x7ff0740c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 >> 172.21.15.62:0/593217541 conn(0x7ff074071ae0 msgr2=0x7ff0740b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 shutdown_connections 2024-03-28T18:00:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.087+0000 7ff073e00640 1 -- 172.21.15.62:0/593217541 wait complete. 2024-03-28T18:00:01.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T17:58:51.658366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T17:58:52.469560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:00:01.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:00:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.325+0000 7f186b200640 1 Processor -- start 2024-03-28T18:00:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.326+0000 7f186b200640 1 -- start start 2024-03-28T18:00:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.327+0000 7f186b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 0x7f186c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.327+0000 7f186b200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f186c15e2c0 0x7f186c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.327+0000 7f186b200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0cf570 con 0x7f186c0b3210 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.327+0000 7f186b200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c161bb0 con 0x7f186c15e2c0 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.327+0000 7f186b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c161e90 con 0x7f186c0cec50 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.328+0000 7f1869400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f186c15e2c0 0x7f186c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.328+0000 7f1869e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 0x7f186c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.328+0000 7f1869400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f186c15e2c0 0x7f186c0cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34730/0 (socket says 172.21.15.62:34730) 2024-03-28T18:00:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.328+0000 7f1869400640 1 -- 172.21.15.62:0/1465339550 learned_addr learned my addr 172.21.15.62:0/1465339550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:01.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.329+0000 7f1869400640 1 -- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 msgr2=0x7f186c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:01.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.329+0000 7f1869400640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 0x7f186c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.329+0000 7f1869400640 1 -- 172.21.15.62:0/1465339550 >> v1:172.21.15.81:6789/0 conn(0x7f186c0b3210 legacy=0x7f186c0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.329+0000 7f1869400640 1 -- 172.21.15.62:0/1465339550 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c162170 con 0x7f186c15e2c0 2024-03-28T18:00:01.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.329+0000 7f1869e00640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 0x7f186c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:00:01.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.330+0000 7f1869400640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c15e2c0 0x7f186c0cf030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1854009870 tx=0x7f185402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359be214c3c0d35c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f1868a00640 1 -- 172.21.15.62:0/1465339550 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854034070 con 0x7f186c15e2c0 2024-03-28T18:00:01.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f1868a00640 1 -- 172.21.15.62:0/1465339550 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854030840 con 0x7f186c15e2c0 2024-03-28T18:00:01.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f1868a00640 1 -- 172.21.15.62:0/1465339550 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854030c10 con 0x7f186c15e2c0 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 -- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c15e2c0 msgr2=0x7f186c0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c15e2c0 0x7f186c0cf030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1854009870 tx=0x7f185402b980 comp rx=0 tx=0).stop 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 -- 172.21.15.62:0/1465339550 shutdown_connections 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c15e2c0 0x7f186c0cf030 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 --2- 172.21.15.62:0/1465339550 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0cec50 0x7f186c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 -- 172.21.15.62:0/1465339550 >> 172.21.15.62:0/1465339550 conn(0x7f186c066fb0 msgr2=0x7f186c0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 -- 172.21.15.62:0/1465339550 shutdown_connections 2024-03-28T18:00:01.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.331+0000 7f186b200640 1 -- 172.21.15.62:0/1465339550 wait complete. 2024-03-28T18:00:01.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.333+0000 7f186b200640 1 Processor -- start 2024-03-28T18:00:01.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.333+0000 7f186b200640 1 -- start start 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0b3210 0x7f186c1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c0cb770 0x7f186c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c162a10 con 0x7f186c0cec50 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c162730 con 0x7f186c0cb770 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.334+0000 7f186b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0d2fc0 con 0x7f186c0b3210 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49230/0 (socket says 172.21.15.62:49230) 2024-03-28T18:00:01.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 -- 172.21.15.62:0/2773828094 learned_addr learned my addr 172.21.15.62:0/2773828094 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869e00640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0b3210 0x7f186c1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f186a800640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c0cb770 0x7f186c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 -- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0b3210 msgr2=0x7f186c1e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0b3210 0x7f186c1e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 -- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c0cb770 msgr2=0x7f186c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c0cb770 0x7f186c15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 -- 172.21.15.62:0/2773828094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0ca820 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.335+0000 7f1869400640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f18540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.336+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854034040 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.336+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854039d90 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.336+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c1e7ea0 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.336+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854043630 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.336+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c1e8460 con 0x7f186c0cec50 2024-03-28T18:00:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.337+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c04f400 con 0x7f186c0cec50 2024-03-28T18:00:01.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.338+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f185404a060 con 0x7f186c0cec50 2024-03-28T18:00:01.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.339+0000 7f1862000640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 0x7f184806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.339+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f18540acae0 con 0x7f186c0cec50 2024-03-28T18:00:01.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.340+0000 7f1869e00640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 0x7f184806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.341+0000 7f1869e00640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 0x7f184806e090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f185c009560 tx=0x7f185c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.344+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f185403e070 con 0x7f186c0cec50 2024-03-28T18:00:01.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.539+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f186c06b8d0 con 0x7f186c0cec50 2024-03-28T18:00:01.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.540+0000 7f1862000640 1 -- 172.21.15.62:0/2773828094 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f185407fe20 con 0x7f186c0cec50 2024-03-28T18:00:01.540 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 msgr2=0x7f184806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 0x7f184806e090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f185c009560 tx=0x7f185c008040 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 msgr2=0x7f186c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f18540046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 shutdown_connections 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f184806bbd0 0x7f184806e090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f186c0cb770 0x7f186c15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f186c0cec50 0x7f186c0ca2e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 --2- 172.21.15.62:0/2773828094 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f186c0b3210 0x7f186c1e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 >> 172.21.15.62:0/2773828094 conn(0x7f186c066fb0 msgr2=0x7f186c0cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:01.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 shutdown_connections 2024-03-28T18:00:01.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.543+0000 7f186b200640 1 -- 172.21.15.62:0/2773828094 wait complete. 2024-03-28T18:00:01.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:00:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.774+0000 7f9fa9200640 1 Processor -- start 2024-03-28T18:00:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.774+0000 7f9fa9200640 1 -- start start 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa9200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9fa40b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa9200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa40d0050 con 0x7f9fa40732e0 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa9200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa40cfd70 con 0x7f9fa40ad8e0 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa9200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4074480 con 0x7f9fa415e2c0 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa1400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9fa40732e0 0x7f9fa40ad1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33858/0 (socket says 172.21.15.62:33858) 2024-03-28T18:00:01.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa1400640 1 -- 172.21.15.62:0/965890181 learned_addr learned my addr 172.21.15.62:0/965890181 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.776+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9fa40b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224958808 0 0) 0x7f9fa40d0050 con 0x7f9fa40732e0 2024-03-28T18:00:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f88003520 con 0x7f9fa40732e0 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2352082738 0 0) 0x7f9f88003520 con 0x7f9fa40732e0 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 msgr2=0x7f9fa40b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9fa40b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 >> v1:172.21.15.81:6789/0 conn(0x7f9fa40ad8e0 legacy=0x7f9fa40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4074760 con 0x7f9fa40732e0 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f9f8c001670 con 0x7f9fa40732e0 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9fa40b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 >> v1:172.21.15.62:6789/0 conn(0x7f9fa40732e0 legacy=0x7f9fa40ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f880039e0 0x7f9f88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9f88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f88004d40 0x7f9f880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4074480 con 0x7f9f88004d40 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800ba70 con 0x7f9f880039e0 2024-03-28T18:00:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.777+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800bc20 con 0x7f9fa415e2c0 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f88004d40 0x7f9f880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa1400640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9f88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f880039e0 0x7f9f88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 -- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 msgr2=0x7f9f88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa415e2c0 0x7f9f88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 -- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f880039e0 msgr2=0x7f9f88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f880039e0 0x7f9f88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.778+0000 7f9fa2800640 1 -- 172.21.15.62:0/965890181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88003520 con 0x7f9f88004d40 2024-03-28T18:00:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.779+0000 7f9fa2800640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f88004d40 0x7f9f880051c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001750 tx=0x7f9f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c089789900767d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c001370 con 0x7f9f88004d40 2024-03-28T18:00:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0052d0 con 0x7f9f88004d40 2024-03-28T18:00:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa0a00640 1 -- 172.21.15.62:0/965890181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c003830 con 0x7f9f88004d40 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa9200640 1 -- 172.21.15.62:0/965890181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f88004d40 msgr2=0x7f9f880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa9200640 1 --2- 172.21.15.62:0/965890181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f88004d40 0x7f9f880051c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001750 tx=0x7f9f88005830 comp rx=0 tx=0).stop 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa2800640 1 -- 172.21.15.62:0/965890181 reap_dead start 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa9200640 1 -- 172.21.15.62:0/965890181 shutdown_connections 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa9200640 1 -- 172.21.15.62:0/965890181 >> 172.21.15.62:0/965890181 conn(0x7f9fa40727c0 msgr2=0x7f9fa4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.780+0000 7f9fa9200640 1 -- 172.21.15.62:0/965890181 shutdown_connections 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.781+0000 7f9fa9200640 1 -- 172.21.15.62:0/965890181 wait complete. 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.782+0000 7f9fa9200640 1 Processor -- start 2024-03-28T18:00:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.782+0000 7f9fa9200640 1 -- start start 2024-03-28T18:00:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f88004d40 0x7f9fa415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa40b5390 0x7f9fa4152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40add70 con 0x7f9fa4151960 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa415e6a0 con 0x7f9f88004d40 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.783+0000 7f9fa9200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40adef0 con 0x7f9fa40b5390 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49258/0 (socket says 172.21.15.62:49258) 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 -- 172.21.15.62:0/2330973517 learned_addr learned my addr 172.21.15.62:0/2330973517 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa2800640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa40b5390 0x7f9fa4152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f88004d40 0x7f9fa415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 -- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa40b5390 msgr2=0x7f9fa4152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa40b5390 0x7f9fa4152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 -- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f88004d40 msgr2=0x7f9fa415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f88004d40 0x7f9fa415fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.784+0000 7f9fa1400640 1 -- 172.21.15.62:0/2330973517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa40b5880 con 0x7f9fa4151960 2024-03-28T18:00:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9fa1400640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c001eb0 tx=0x7f9f8c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00c8a0 con 0x7f9fa4151960 2024-03-28T18:00:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c018070 con 0x7f9fa4151960 2024-03-28T18:00:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00cc80 con 0x7f9fa4151960 2024-03-28T18:00:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa40b5b60 con 0x7f9fa4151960 2024-03-28T18:00:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.785+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa40b2fe0 con 0x7f9fa4151960 2024-03-28T18:00:01.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.786+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa40680c0 con 0x7f9fa4151960 2024-03-28T18:00:01.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.787+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9f8c01d460 con 0x7f9fa4151960 2024-03-28T18:00:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.788+0000 7f9f9a000640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 0x7f9f8006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.788+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9f8c08aad0 con 0x7f9fa4151960 2024-03-28T18:00:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.789+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 0x7f9f8006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.790+0000 7f9fa1e00640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 0x7f9f8006e090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009bd0 tx=0x7f9f9400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.793+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9f8c00ce20 con 0x7f9fa4151960 2024-03-28T18:00:01.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.994+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fa4161b50 con 0x7f9fa4151960 2024-03-28T18:00:01.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.994+0000 7f9f9a000640 1 -- 172.21.15.62:0/2330973517 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f8c0594a0 con 0x7f9fa4151960 2024-03-28T18:00:01.995 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 msgr2=0x7f9f8006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 0x7f9f8006e090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009bd0 tx=0x7f9f9400b040 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 msgr2=0x7f9fa40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c001eb0 tx=0x7f9f8c00ec60 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 shutdown_connections 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9f8006bbd0 0x7f9f8006e090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9fa40b5390 0x7f9fa4152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa4151960 0x7f9fa40b4e50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 --2- 172.21.15.62:0/2330973517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f88004d40 0x7f9fa415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:01.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 >> 172.21.15.62:0/2330973517 conn(0x7f9fa40727c0 msgr2=0x7f9fa40af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:02.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 shutdown_connections 2024-03-28T18:00:02.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:01.999+0000 7f9fa9200640 1 -- 172.21.15.62:0/2330973517 wait complete. 2024-03-28T18:00:02.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:00:02.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.237+0000 7f2bf6800640 1 Processor -- start 2024-03-28T18:00:02.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.238+0000 7f2bf6800640 1 -- start start 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf6800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00afa50 0x7f2bf00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf6800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2bf00675c0 0x7f2bf00cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf6800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf0067be0 con 0x7f2bf00b62e0 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf6800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf00afe30 con 0x7f2bf00675c0 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf6800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0160470 con 0x7f2bf00afa50 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf4a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2bf00b62e0 0x7f2bf0066eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33868/0 (socket says 172.21.15.62:33868) 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf4a00640 1 -- 172.21.15.62:0/2901918563 learned_addr learned my addr 172.21.15.62:0/2901918563 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.240+0000 7f2bf5e00640 1 --2- 172.21.15.62:0/2901918563 >> v2:172.21.15.62:3300/0 conn(0x7f2bf00675c0 0x7f2bf00cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2bf5400640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00afa50 0x7f2bf00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2bf5e00640 1 -- 172.21.15.62:0/2901918563 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00afa50 msgr2=0x7f2bf00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2bf5e00640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00afa50 0x7f2bf00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2befe00640 1 -- 172.21.15.62:0/2901918563 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054537599 0 0) 0x7f2bf0067be0 con 0x7f2bf00b62e0 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2bf5e00640 1 -- 172.21.15.62:0/2901918563 >> v1:172.21.15.62:6789/0 conn(0x7f2bf00b62e0 legacy=0x7f2bf0066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.241+0000 7f2bf5e00640 1 -- 172.21.15.62:0/2901918563 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf015f6b0 con 0x7f2bf00675c0 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.242+0000 7f2bf5e00640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00675c0 0x7f2bf00cdad0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009d90 tx=0x7f2be802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f83c01ac797670 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.242+0000 7f2befe00640 1 -- 172.21.15.62:0/2901918563 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8039070 con 0x7f2bf00675c0 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.242+0000 7f2befe00640 1 -- 172.21.15.62:0/2901918563 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be802cbd0 con 0x7f2bf00675c0 2024-03-28T18:00:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.242+0000 7f2befe00640 1 -- 172.21.15.62:0/2901918563 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8034440 con 0x7f2bf00675c0 2024-03-28T18:00:02.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 -- 172.21.15.62:0/2901918563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00675c0 msgr2=0x7f2bf00cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00675c0 0x7f2bf00cdad0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009d90 tx=0x7f2be802be20 comp rx=0 tx=0).stop 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 -- 172.21.15.62:0/2901918563 shutdown_connections 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00675c0 0x7f2bf00cdad0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 --2- 172.21.15.62:0/2901918563 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00afa50 0x7f2bf00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 -- 172.21.15.62:0/2901918563 >> 172.21.15.62:0/2901918563 conn(0x7f2bf0071970 msgr2=0x7f2bf0071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 -- 172.21.15.62:0/2901918563 shutdown_connections 2024-03-28T18:00:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.243+0000 7f2bf6800640 1 -- 172.21.15.62:0/2901918563 wait complete. 2024-03-28T18:00:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.245+0000 7f2bf6800640 1 Processor -- start 2024-03-28T18:00:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.245+0000 7f2bf6800640 1 -- start start 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00675c0 0x7f2bf00aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 0x7f2bf00b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf015fc70 con 0x7f2bf00afa50 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf00b6720 con 0x7f2bf00b62e0 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf6800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf015ff50 con 0x7f2bf00675c0 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49280/0 (socket says 172.21.15.62:49280) 2024-03-28T18:00:02.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.246+0000 7f2bf4a00640 1 -- 172.21.15.62:0/1577370237 learned_addr learned my addr 172.21.15.62:0/1577370237 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf5400640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00675c0 0x7f2bf00aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf5e00640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 0x7f2bf00b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 -- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00675c0 msgr2=0x7f2bf00aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00675c0 0x7f2bf00aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 -- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 msgr2=0x7f2bf00b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 0x7f2bf00b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 -- 172.21.15.62:0/1577370237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf01531c0 con 0x7f2bf00afa50 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf5e00640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 0x7f2bf00b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:02.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf4a00640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c460 tx=0x7f2bdc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc018070 con 0x7f2bf00afa50 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc007500 con 0x7f2bf00afa50 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf01534a0 con 0x7f2bf00afa50 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.247+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004a20 con 0x7f2bf00afa50 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.248+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf0153a60 con 0x7f2bf00afa50 2024-03-28T18:00:02.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.249+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf004f400 con 0x7f2bf00afa50 2024-03-28T18:00:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.250+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2bdc01b070 con 0x7f2bf00afa50 2024-03-28T18:00:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.251+0000 7f2bed600640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 0x7f2bd006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.251+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2bdc089e30 con 0x7f2bf00afa50 2024-03-28T18:00:02.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.251+0000 7f2bf5400640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 0x7f2bd006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.252+0000 7f2bf5400640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 0x7f2bd006e0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2be4006fd0 tx=0x7f2be4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.255+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2bdc058800 con 0x7f2bf00afa50 2024-03-28T18:00:02.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.459+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bf00728d0 con 0x7f2bf00afa50 2024-03-28T18:00:02.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.460+0000 7f2bed600640 1 -- 172.21.15.62:0/1577370237 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bdc058100 con 0x7f2bf00afa50 2024-03-28T18:00:02.460 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 msgr2=0x7f2bd006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 0x7f2bd006e0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2be4006fd0 tx=0x7f2be4008040 comp rx=0 tx=0).stop 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 msgr2=0x7f2bf00b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c460 tx=0x7f2bdc00c920 comp rx=0 tx=0).stop 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 shutdown_connections 2024-03-28T18:00:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2bd006bc00 0x7f2bd006e0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf00b62e0 0x7f2bf00b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bf00afa50 0x7f2bf00b4be0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 --2- 172.21.15.62:0/1577370237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2bf00675c0 0x7f2bf00aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 >> 172.21.15.62:0/1577370237 conn(0x7f2bf0071970 msgr2=0x7f2bf006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.463+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 shutdown_connections 2024-03-28T18:00:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.464+0000 7f2bf6800640 1 -- 172.21.15.62:0/1577370237 wait complete. 2024-03-28T18:00:02.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:02.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.710+0000 7efef5200640 1 Processor -- start 2024-03-28T18:00:02.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.711+0000 7efef5200640 1 -- start start 2024-03-28T18:00:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.712+0000 7efef5200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.712+0000 7efef5200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 0x7efef0071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.712+0000 7efef5200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00b3700 con 0x7efef00b31f0 2024-03-28T18:00:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.712+0000 7efef5200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00b39e0 con 0x7efef015ef60 2024-03-28T18:00:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.712+0000 7efef5200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef00b3cc0 con 0x7efef0144740 2024-03-28T18:00:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.713+0000 7efeede00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.713+0000 7efeee800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 0x7efef0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.713+0000 7efeee800640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 0x7efef0071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34774/0 (socket says 172.21.15.62:34774) 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.713+0000 7efeee800640 1 -- 172.21.15.62:0/896050668 learned_addr learned my addr 172.21.15.62:0/896050668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.714+0000 7efeede00640 1 -- 172.21.15.62:0/896050668 >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 msgr2=0x7efef0071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.714+0000 7efeede00640 1 --2- 172.21.15.62:0/896050668 >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 0x7efef0071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.714+0000 7efeede00640 1 -- 172.21.15.62:0/896050668 >> v1:172.21.15.81:6789/0 conn(0x7efef00b31f0 legacy=0x7efef015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.714+0000 7efeede00640 1 -- 172.21.15.62:0/896050668 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00b3fa0 con 0x7efef0144740 2024-03-28T18:00:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.715+0000 7efeede00640 1 --2- 172.21.15.62:0/896050668 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b2cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efee00097a0 tx=0x7efee002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d6d3497c37c104f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.716+0000 7efeed400640 1 -- 172.21.15.62:0/896050668 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0035020 con 0x7efef0144740 2024-03-28T18:00:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.716+0000 7efeed400640 1 -- 172.21.15.62:0/896050668 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0031810 con 0x7efef0144740 2024-03-28T18:00:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.716+0000 7efeed400640 1 -- 172.21.15.62:0/896050668 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0031be0 con 0x7efef0144740 2024-03-28T18:00:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 -- 172.21.15.62:0/896050668 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 msgr2=0x7efef00b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 --2- 172.21.15.62:0/896050668 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b2cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efee00097a0 tx=0x7efee002b8b0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 -- 172.21.15.62:0/896050668 shutdown_connections 2024-03-28T18:00:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 --2- 172.21.15.62:0/896050668 >> v2:172.21.15.81:3300/0 conn(0x7efef015ef60 0x7efef0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 --2- 172.21.15.62:0/896050668 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b2cb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 -- 172.21.15.62:0/896050668 >> 172.21.15.62:0/896050668 conn(0x7efef0066fb0 msgr2=0x7efef00673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 -- 172.21.15.62:0/896050668 shutdown_connections 2024-03-28T18:00:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.717+0000 7efef5200640 1 -- 172.21.15.62:0/896050668 wait complete. 2024-03-28T18:00:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.719+0000 7efef5200640 1 Processor -- start 2024-03-28T18:00:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.719+0000 7efef5200640 1 -- start start 2024-03-28T18:00:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 0x7efef00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00716e0 con 0x7efef00b31f0 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0071560 con 0x7efef015ef60 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efef5200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0071860 con 0x7efef0144740 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efeede00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efeede00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49296/0 (socket says 172.21.15.62:49296) 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.720+0000 7efeede00640 1 -- 172.21.15.62:0/3023608651 learned_addr learned my addr 172.21.15.62:0/3023608651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efee7e00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeee800640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 0x7efef00c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 -- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 msgr2=0x7efef00b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 -- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 msgr2=0x7efef00c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 0x7efef00c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 -- 172.21.15.62:0/3023608651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00c9880 con 0x7efef00b31f0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeee800640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 0x7efef00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.721+0000 7efeede00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.722+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0041070 con 0x7efef00b31f0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.722+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee003ad00 con 0x7efef00b31f0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.722+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0035040 con 0x7efef00b31f0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.722+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef00c9b60 con 0x7efef00b31f0 2024-03-28T18:00:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.722+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef00ca120 con 0x7efef00b31f0 2024-03-28T18:00:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.723+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef00680c0 con 0x7efef00b31f0 2024-03-28T18:00:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.724+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efee0038020 con 0x7efef00b31f0 2024-03-28T18:00:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.725+0000 7efee6000640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 0x7efecc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.725+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efee00ac750 con 0x7efef00b31f0 2024-03-28T18:00:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.725+0000 7efee7e00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 0x7efecc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:02.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.726+0000 7efee7e00640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 0x7efecc06e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efef00b0a90 tx=0x7efed8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.729+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7efee007b120 con 0x7efef00b31f0 2024-03-28T18:00:02.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.972+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef00c9570 con 0x7efef00b31f0 2024-03-28T18:00:02.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.974+0000 7efee6000640 1 -- 172.21.15.62:0/3023608651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7efee007aa20 con 0x7efef00b31f0 2024-03-28T18:00:02.974 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:02.974 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T18:00:02.975 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 msgr2=0x7efecc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 0x7efecc06e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efef00b0a90 tx=0x7efed8009040 comp rx=0 tx=0).stop 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 msgr2=0x7efef013e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee00046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 shutdown_connections 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7efecc06bbd0 0x7efecc06e090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efef015ef60 0x7efef00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7efef0144740 0x7efef00b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.976+0000 7efef5200640 1 --2- 172.21.15.62:0/3023608651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efef00b31f0 0x7efef013e630 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:02.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.977+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 >> 172.21.15.62:0/3023608651 conn(0x7efef0066fb0 msgr2=0x7efef015b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:02.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.977+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 shutdown_connections 2024-03-28T18:00:02.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:02.977+0000 7efef5200640 1 -- 172.21.15.62:0/3023608651 wait complete. 2024-03-28T18:00:02.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:03.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.213+0000 7f970b800640 1 Processor -- start 2024-03-28T18:00:03.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.213+0000 7f970b800640 1 -- start start 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f970b800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9704152f90 0x7f970406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f970b800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 0x7f97040cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f970b800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97041603c0 con 0x7f970415feb0 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f970b800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97041606a0 con 0x7f9704152f90 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f970b800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040b3cb0 con 0x7f970415fa00 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f9708e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9704152f90 0x7f970406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f9708e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9704152f90 0x7f970406ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49304/0 (socket says 172.21.15.62:49304) 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.215+0000 7f9708e00640 1 -- 172.21.15.62:0/3110889290 learned_addr learned my addr 172.21.15.62:0/3110889290 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9703e00640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 0x7f97040cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9708e00640 1 -- 172.21.15.62:0/3110889290 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 msgr2=0x7f97040cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9708e00640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 0x7f97040cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9708e00640 1 -- 172.21.15.62:0/3110889290 >> v1:172.21.15.81:6789/0 conn(0x7f970415feb0 legacy=0x7f97040b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:03.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9708e00640 1 -- 172.21.15.62:0/3110889290 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040b3f90 con 0x7f9704152f90 2024-03-28T18:00:03.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.216+0000 7f9703e00640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 0x7f97040cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.217+0000 7f9708e00640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9704152f90 0x7f970406ccb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8cf1cbb47b1306 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.217+0000 7f9703400640 1 -- 172.21.15.62:0/3110889290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4035020 con 0x7f9704152f90 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.217+0000 7f9703400640 1 -- 172.21.15.62:0/3110889290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4031640 con 0x7f9704152f90 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.217+0000 7f9703400640 1 -- 172.21.15.62:0/3110889290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f402d040 con 0x7f9704152f90 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 -- 172.21.15.62:0/3110889290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9704152f90 msgr2=0x7f970406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9704152f90 0x7f970406ccb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402b8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 -- 172.21.15.62:0/3110889290 shutdown_connections 2024-03-28T18:00:03.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f970415fa00 0x7f97040cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 --2- 172.21.15.62:0/3110889290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9704152f90 0x7f970406ccb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 -- 172.21.15.62:0/3110889290 >> 172.21.15.62:0/3110889290 conn(0x7f9704066fb0 msgr2=0x7f97040673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:03.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 -- 172.21.15.62:0/3110889290 shutdown_connections 2024-03-28T18:00:03.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.218+0000 7f970b800640 1 -- 172.21.15.62:0/3110889290 wait complete. 2024-03-28T18:00:03.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.220+0000 7f970b800640 1 Processor -- start 2024-03-28T18:00:03.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.220+0000 7f970b800640 1 -- start start 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 0x7f9704150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 0x7f97040cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cf540 con 0x7f970415fa00 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cf3c0 con 0x7f970415feb0 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f970b800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cf6c0 con 0x7f9704152f90 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f9703e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f9703e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49316/0 (socket says 172.21.15.62:49316) 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.221+0000 7f9703e00640 1 -- 172.21.15.62:0/2038924754 learned_addr learned my addr 172.21.15.62:0/2038924754 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9708e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 0x7f9704150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9709800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 0x7f97040cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 -- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 msgr2=0x7f9704150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 0x7f9704150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 -- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 msgr2=0x7f97040cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 0x7f97040cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 -- 172.21.15.62:0/2038924754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970409d340 con 0x7f970415fa00 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9708e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 0x7f9704150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9709800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 0x7f97040cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9703e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c7f0 tx=0x7f96f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8015070 con 0x7f970415fa00 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8007a30 con 0x7f970415fa00 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8007e00 con 0x7f970415fa00 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970409d620 con 0x7f970415fa00 2024-03-28T18:00:03.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.222+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9704150c00 con 0x7f970415fa00 2024-03-28T18:00:03.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.223+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970409d840 con 0x7f970415fa00 2024-03-28T18:00:03.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.225+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f96f8018070 con 0x7f970415fa00 2024-03-28T18:00:03.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.226+0000 7f9700c00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 0x7f96e806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.226+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f96f808aa30 con 0x7f970415fa00 2024-03-28T18:00:03.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.226+0000 7f9708e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 0x7f96e806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.227+0000 7f9708e00640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 0x7f96e806e0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96f4036000 tx=0x7f96f4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.230+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f96f80594a0 con 0x7f970415fa00 2024-03-28T18:00:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.433+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97040680c0 con 0x7f970415fa00 2024-03-28T18:00:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.436+0000 7f9700c00640 1 -- 172.21.15.62:0/2038924754 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f96f8058da0 con 0x7f970415fa00 2024-03-28T18:00:03.436 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:03.436 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 msgr2=0x7f96e806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 0x7f96e806e0c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96f4036000 tx=0x7f96f4008030 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 msgr2=0x7f97040cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c7f0 tx=0x7f96f800cbf0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 shutdown_connections 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f96e806bc00 0x7f96e806e0c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f970415feb0 0x7f97040cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f970415fa00 0x7f97040cab90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 --2- 172.21.15.62:0/2038924754 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9704152f90 0x7f9704150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.438+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 >> 172.21.15.62:0/2038924754 conn(0x7f9704066fb0 msgr2=0x7f97040b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.439+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 shutdown_connections 2024-03-28T18:00:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.439+0000 7f970b800640 1 -- 172.21.15.62:0/2038924754 wait complete. 2024-03-28T18:00:03.459 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T18:00:03.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.685+0000 7f9d95800640 1 Processor -- start 2024-03-28T18:00:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.685+0000 7f9d95800640 1 -- start start 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d95800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d90070f50 0x7f9d9015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d95800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d95800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900713d0 con 0x7f9d901599d0 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d95800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900d11a0 con 0x7f9d90070f50 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d95800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900d1480 con 0x7f9d900ce520 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d8f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d90070f50 0x7f9d9015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d8f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d90070f50 0x7f9d9015cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49320/0 (socket says 172.21.15.62:49320) 2024-03-28T18:00:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d8f400640 1 -- 172.21.15.62:0/2955782137 learned_addr learned my addr 172.21.15.62:0/2955782137 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.687+0000 7f9d8fe00640 1 --1- 172.21.15.62:0/2955782137 >> v1:172.21.15.62:6789/0 conn(0x7f9d901599d0 0x7f9d900d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33882/0 (socket says 172.21.15.62:33882) 2024-03-28T18:00:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d94e00640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d8f400640 1 -- 172.21.15.62:0/2955782137 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 msgr2=0x7f9d900ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d8f400640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d8f400640 1 -- 172.21.15.62:0/2955782137 >> v1:172.21.15.62:6789/0 conn(0x7f9d901599d0 legacy=0x7f9d900d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d8f400640 1 -- 172.21.15.62:0/2955782137 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900d1760 con 0x7f9d90070f50 2024-03-28T18:00:03.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.688+0000 7f9d94e00640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.689+0000 7f9d8f400640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d90070f50 0x7f9d9015cd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009870 tx=0x7f9d7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145ed0086818d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.689+0000 7f9d8ea00640 1 -- 172.21.15.62:0/2955782137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c034070 con 0x7f9d90070f50 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.689+0000 7f9d8ea00640 1 -- 172.21.15.62:0/2955782137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c030840 con 0x7f9d90070f50 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.689+0000 7f9d8ea00640 1 -- 172.21.15.62:0/2955782137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c030c10 con 0x7f9d90070f50 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 -- 172.21.15.62:0/2955782137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d90070f50 msgr2=0x7f9d9015cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d90070f50 0x7f9d9015cd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009870 tx=0x7f9d7c02b980 comp rx=0 tx=0).stop 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 -- 172.21.15.62:0/2955782137 shutdown_connections 2024-03-28T18:00:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 --2- 172.21.15.62:0/2955782137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d90070f50 0x7f9d9015cd00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 -- 172.21.15.62:0/2955782137 >> 172.21.15.62:0/2955782137 conn(0x7f9d9006ec40 msgr2=0x7f9d9006f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 -- 172.21.15.62:0/2955782137 shutdown_connections 2024-03-28T18:00:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.690+0000 7f9d95800640 1 -- 172.21.15.62:0/2955782137 wait complete. 2024-03-28T18:00:03.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.692+0000 7f9d95800640 1 Processor -- start 2024-03-28T18:00:03.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.692+0000 7f9d95800640 1 -- start start 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 0x7f9d900c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 0x7f9d90144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900cc2e0 con 0x7f9d901599d0 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900cc160 con 0x7f9d90070f50 2024-03-28T18:00:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d95800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900cc460 con 0x7f9d900ce520 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 0x7f9d900c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.693+0000 7f9d94e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 0x7f9d90144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 0x7f9d900c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34812/0 (socket says 172.21.15.62:34812) 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d8fe00640 1 -- 172.21.15.62:0/2026768349 learned_addr learned my addr 172.21.15.62:0/2026768349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 -- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 msgr2=0x7f9d900ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 -- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 msgr2=0x7f9d900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 0x7f9d900c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 -- 172.21.15.62:0/2026768349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900caba0 con 0x7f9d901599d0 2024-03-28T18:00:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.694+0000 7f9d94e00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 0x7f9d90144430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800cdf0 tx=0x7f9d8800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.695+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8801a070 con 0x7f9d901599d0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.695+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8800d840 con 0x7f9d901599d0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.695+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800dc70 con 0x7f9d901599d0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.695+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900cae80 con 0x7f9d901599d0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.695+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d90144970 con 0x7f9d901599d0 2024-03-28T18:00:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.696+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d900cb3a0 con 0x7f9d901599d0 2024-03-28T18:00:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.697+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9d8801d070 con 0x7f9d901599d0 2024-03-28T18:00:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.698+0000 7f9d87e00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 0x7f9d7006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.698+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9d8808b1a0 con 0x7f9d901599d0 2024-03-28T18:00:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.699+0000 7f9d8fe00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 0x7f9d7006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:03.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.700+0000 7f9d8fe00640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 0x7f9d7006e090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d80004de0 tx=0x7f9d80007f10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.703+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f9d88059650 con 0x7f9d901599d0 2024-03-28T18:00:03.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.942+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d90070170 con 0x7f9d901599d0 2024-03-28T18:00:03.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.943+0000 7f9d87e00640 1 -- 172.21.15.62:0/2026768349 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7f9d8805d6f0 con 0x7f9d901599d0 2024-03-28T18:00:03.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:03.944 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T18:00:03.944 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T18:00:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 msgr2=0x7f9d7006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 0x7f9d7006e090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d80004de0 tx=0x7f9d80007f10 comp rx=0 tx=0).stop 2024-03-28T18:00:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 msgr2=0x7f9d90144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 0x7f9d90144430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800cdf0 tx=0x7f9d8800a710 comp rx=0 tx=0).stop 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 shutdown_connections 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f9d7006bbd0 0x7f9d7006e090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901599d0 0x7f9d90144430 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f9d900ce520 0x7f9d900ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 --2- 172.21.15.62:0/2026768349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90070f50 0x7f9d900c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.945+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 >> 172.21.15.62:0/2026768349 conn(0x7f9d9006ec40 msgr2=0x7f9d90157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.946+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 shutdown_connections 2024-03-28T18:00:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:03.946+0000 7f9d95800640 1 -- 172.21.15.62:0/2026768349 wait complete. 2024-03-28T18:00:03.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.182+0000 7fd640a00640 1 Processor -- start 2024-03-28T18:00:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.183+0000 7fd640a00640 1 -- start start 2024-03-28T18:00:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.184+0000 7fd640a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd63c15dd60 0x7fd63c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.184+0000 7fd640a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 0x7fd63c0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.184+0000 7fd640a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0b6e60 con 0x7fd63c0b5c30 2024-03-28T18:00:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.184+0000 7fd640a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0b60a0 con 0x7fd63c15dd60 2024-03-28T18:00:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.184+0000 7fd640a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0b6c30 con 0x7fd63c0b99f0 2024-03-28T18:00:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd63b200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd63c0b5c30 0x7fd63c161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52224/0 (socket says 172.21.15.62:52224) 2024-03-28T18:00:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd63b200640 1 -- 172.21.15.62:0/3203262255 learned_addr learned my addr 172.21.15.62:0/3203262255 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd63a800640 1 --2- 172.21.15.62:0/3203262255 >> v2:172.21.15.81:3300/0 conn(0x7fd63c15dd60 0x7fd63c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd639e00640 1 --2- 172.21.15.62:0/3203262255 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 0x7fd63c0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd639400640 1 -- 172.21.15.62:0/3203262255 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766009714 0 0) 0x7fd63c0b6e60 con 0x7fd63c0b5c30 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.185+0000 7fd639400640 1 -- 172.21.15.62:0/3203262255 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c003520 con 0x7fd63c0b5c30 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.186+0000 7fd639e00640 1 -- 172.21.15.62:0/3203262255 >> v2:172.21.15.81:3300/0 conn(0x7fd63c15dd60 msgr2=0x7fd63c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.186+0000 7fd639e00640 1 --2- 172.21.15.62:0/3203262255 >> v2:172.21.15.81:3300/0 conn(0x7fd63c15dd60 0x7fd63c0b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.186+0000 7fd639e00640 1 -- 172.21.15.62:0/3203262255 >> v1:172.21.15.62:6789/0 conn(0x7fd63c0b5c30 legacy=0x7fd63c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.186+0000 7fd639e00640 1 -- 172.21.15.62:0/3203262255 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0b7100 con 0x7fd63c0b99f0 2024-03-28T18:00:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.187+0000 7fd639e00640 1 --2- 172.21.15.62:0/3203262255 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 0x7fd63c0b56f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd624009d90 tx=0x7fd62402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3627bae9fde23f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.187+0000 7fd639400640 1 -- 172.21.15.62:0/3203262255 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624039070 con 0x7fd63c0b99f0 2024-03-28T18:00:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd639400640 1 -- 172.21.15.62:0/3203262255 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62402cce0 con 0x7fd63c0b99f0 2024-03-28T18:00:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd639400640 1 -- 172.21.15.62:0/3203262255 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624034640 con 0x7fd63c0b99f0 2024-03-28T18:00:04.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 -- 172.21.15.62:0/3203262255 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 msgr2=0x7fd63c0b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 --2- 172.21.15.62:0/3203262255 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 0x7fd63c0b56f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd624009d90 tx=0x7fd62402bf30 comp rx=0 tx=0).stop 2024-03-28T18:00:04.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 -- 172.21.15.62:0/3203262255 shutdown_connections 2024-03-28T18:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 --2- 172.21.15.62:0/3203262255 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c0b99f0 0x7fd63c0b56f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 --2- 172.21.15.62:0/3203262255 >> v2:172.21.15.81:3300/0 conn(0x7fd63c15dd60 0x7fd63c0b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 -- 172.21.15.62:0/3203262255 >> 172.21.15.62:0/3203262255 conn(0x7fd63c071ae0 msgr2=0x7fd63c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 -- 172.21.15.62:0/3203262255 shutdown_connections 2024-03-28T18:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.188+0000 7fd640a00640 1 -- 172.21.15.62:0/3203262255 wait complete. 2024-03-28T18:00:04.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.190+0000 7fd640a00640 1 Processor -- start 2024-03-28T18:00:04.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.190+0000 7fd640a00640 1 -- start start 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 0x7fd63c160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 0x7fd63c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0b79a0 con 0x7fd63c0b5c30 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0b76c0 con 0x7fd63c0b99f0 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd640a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15e140 con 0x7fd63c15dd60 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd63a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd63a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48682/0 (socket says 172.21.15.62:48682) 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.191+0000 7fd63a800640 1 -- 172.21.15.62:0/4193617117 learned_addr learned my addr 172.21.15.62:0/4193617117 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd639e00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 0x7fd63c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63b200640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 0x7fd63c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 -- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 msgr2=0x7fd63c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 0x7fd63c0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 -- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 msgr2=0x7fd63c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 0x7fd63c160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 -- 172.21.15.62:0/4193617117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c15e5f0 con 0x7fd63c0b5c30 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd639e00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 0x7fd63c160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63b200640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 0x7fd63c0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd63a800640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00e870 tx=0x7fd62c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c017070 con 0x7fd63c0b5c30 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c00c940 con 0x7fd63c0b5c30 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00cd30 con 0x7fd63c0b5c30 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.192+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c15e8d0 con 0x7fd63c0b5c30 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.193+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c15ee90 con 0x7fd63c0b5c30 2024-03-28T18:00:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.194+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c072b90 con 0x7fd63c0b5c30 2024-03-28T18:00:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.195+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd62c01a070 con 0x7fd63c0b5c30 2024-03-28T18:00:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.196+0000 7fd632a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 0x7fd61806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.196+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd62c08ad90 con 0x7fd63c0b5c30 2024-03-28T18:00:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.196+0000 7fd639e00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 0x7fd61806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.197+0000 7fd639e00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 0x7fd61806e090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd624036000 tx=0x7fd624030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.201+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd62c059800 con 0x7fd63c0b5c30 2024-03-28T18:00:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.443+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63c04f4e0 con 0x7fd63c0b5c30 2024-03-28T18:00:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.444+0000 7fd632a00640 1 -- 172.21.15.62:0/4193617117 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4971 (secure 0 0 0) 0x7fd62c059100 con 0x7fd63c0b5c30 2024-03-28T18:00:04.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:04.445 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1623084832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1623084832},{"type":"v1","addr":"172.21.15.62:6835","nonce":1623084832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2631605359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2631605359},{"type":"v1","addr":"172.21.15.62:6833","nonce":2631605359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4143709612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":4143709612},{"type":"v1","addr":"172.21.15.81:6836","nonce":4143709612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T17:58:53.451830+0000","modified":"2024-03-28T17:58:56.522056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/3525951990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3525951990},{"type":"v1","addr":"172.21.15.81:6837","nonce":3525951990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-28T18:00:04.445 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-03-28T18:00:04.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 msgr2=0x7fd61806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 0x7fd61806e090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd624036000 tx=0x7fd624030000 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 msgr2=0x7fd63c153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00e870 tx=0x7fd62c00ed30 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 shutdown_connections 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd61806bbd0 0x7fd61806e090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd63c15dd60 0x7fd63c0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd63c0b99f0 0x7fd63c160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.447+0000 7fd640a00640 1 --2- 172.21.15.62:0/4193617117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd63c0b5c30 0x7fd63c153290 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.448+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 >> 172.21.15.62:0/4193617117 conn(0x7fd63c071ae0 msgr2=0x7fd63c161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.448+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 shutdown_connections 2024-03-28T18:00:04.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.448+0000 7fd640a00640 1 -- 172.21.15.62:0/4193617117 wait complete. 2024-03-28T18:00:04.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:04.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.682+0000 7f4189c00640 1 Processor -- start 2024-03-28T18:00:04.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.682+0000 7f4189c00640 1 -- start start 2024-03-28T18:00:04.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.684+0000 7f4189c00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 0x7f418415f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.684+0000 7f4189c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.684+0000 7f4189c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840cebb0 con 0x7f4184072ab0 2024-03-28T18:00:04.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.684+0000 7f4189c00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41840cee90 con 0x7f41841456d0 2024-03-28T18:00:04.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.684+0000 7f4189c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41840cf170 con 0x7f418415fec0 2024-03-28T18:00:04.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.685+0000 7f4189200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.685+0000 7f4183e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 0x7f418415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.685+0000 7f4189200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:60804/0 (socket says 172.21.15.62:60804) 2024-03-28T18:00:04.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.685+0000 7f4183e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 0x7f418415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34542/0 (socket says 172.21.15.62:34542) 2024-03-28T18:00:04.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.685+0000 7f4189200640 1 -- 172.21.15.62:0/909160116 learned_addr learned my addr 172.21.15.62:0/909160116 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:04.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.686+0000 7f4189200640 1 -- 172.21.15.62:0/909160116 >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 msgr2=0x7f418415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:04.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.686+0000 7f4189200640 1 --2- 172.21.15.62:0/909160116 >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 0x7f418415f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.686+0000 7f4189200640 1 -- 172.21.15.62:0/909160116 >> v1:172.21.15.81:6789/0 conn(0x7f4184072ab0 legacy=0x7f418406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.686+0000 7f4189200640 1 -- 172.21.15.62:0/909160116 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840cf450 con 0x7f418415fec0 2024-03-28T18:00:04.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.686+0000 7f4182a00640 1 -- 172.21.15.62:0/909160116 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561571629 0 0) 0x7f41840cebb0 con 0x7f4184072ab0 2024-03-28T18:00:04.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.687+0000 7f4189200640 1 --2- 172.21.15.62:0/909160116 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f417c009e50 tx=0x7f417c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9b0a61c523417a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.688+0000 7f4182a00640 1 -- 172.21.15.62:0/909160116 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c02ca80 con 0x7f418415fec0 2024-03-28T18:00:04.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.688+0000 7f4182a00640 1 -- 172.21.15.62:0/909160116 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c02cc20 con 0x7f418415fec0 2024-03-28T18:00:04.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.688+0000 7f4182a00640 1 -- 172.21.15.62:0/909160116 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0343f0 con 0x7f418415fec0 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.688+0000 7f4189c00640 1 -- 172.21.15.62:0/909160116 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 msgr2=0x7f4184071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.688+0000 7f4189c00640 1 --2- 172.21.15.62:0/909160116 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f417c009e50 tx=0x7f417c02bee0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 -- 172.21.15.62:0/909160116 shutdown_connections 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 --2- 172.21.15.62:0/909160116 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f4184071a50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 --2- 172.21.15.62:0/909160116 >> v2:172.21.15.81:3300/0 conn(0x7f41841456d0 0x7f418415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 -- 172.21.15.62:0/909160116 >> 172.21.15.62:0/909160116 conn(0x7f41840b1730 msgr2=0x7f41840b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 -- 172.21.15.62:0/909160116 shutdown_connections 2024-03-28T18:00:04.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.689+0000 7f4189c00640 1 -- 172.21.15.62:0/909160116 wait complete. 2024-03-28T18:00:04.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.691+0000 7f4189c00640 1 Processor -- start 2024-03-28T18:00:04.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.691+0000 7f4189c00640 1 -- start start 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4184072ab0 0x7f41840b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f418413ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184071f90 con 0x7f41841456d0 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41840cf730 con 0x7f4184072ab0 2024-03-28T18:00:04.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.692+0000 7f4189c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418406d490 con 0x7f418415fec0 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48698/0 (socket says 172.21.15.62:48698) 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 -- 172.21.15.62:0/3910568943 learned_addr learned my addr 172.21.15.62:0/3910568943 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183e00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4184072ab0 0x7f41840b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4189200640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f418413ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 -- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 msgr2=0x7f418413ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f418413ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 -- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4184072ab0 msgr2=0x7f41840b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4184072ab0 0x7f41840b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.693+0000 7f4183400640 1 -- 172.21.15.62:0/3910568943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840b3430 con 0x7f41841456d0 2024-03-28T18:00:04.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f4183400640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f417000c4b0 tx=0x7f417000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170017070 con 0x7f41841456d0 2024-03-28T18:00:04.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170005480 con 0x7f41841456d0 2024-03-28T18:00:04.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170003880 con 0x7f41841456d0 2024-03-28T18:00:04.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41840b3710 con 0x7f41841456d0 2024-03-28T18:00:04.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.694+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418413c880 con 0x7f41841456d0 2024-03-28T18:00:04.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.696+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418413ca80 con 0x7f41841456d0 2024-03-28T18:00:04.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.696+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f417001a070 con 0x7f41841456d0 2024-03-28T18:00:04.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.697+0000 7f417be00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 0x7f416406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:04.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.697+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4170089dd0 con 0x7f41841456d0 2024-03-28T18:00:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.698+0000 7f4183e00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 0x7f416406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.699+0000 7f4183e00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 0x7f416406e0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41840af4f0 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:04.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.702+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f418413ca80 con 0x7f41841456d0 2024-03-28T18:00:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.897+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4184073490 con 0x7f41841456d0 2024-03-28T18:00:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.900+0000 7f417be00640 1 -- 172.21.15.62:0/3910568943 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16807 (secure 0 0 0) 0x7f417008b070 con 0x7f41841456d0 2024-03-28T18:00:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:04.901 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T17:59:36.231803+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T17:58:51.658366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T17:58:52.469560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.902+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 msgr2=0x7f416406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.902+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 0x7f416406e0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41840af4f0 tx=0x7f4174008040 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.902+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 msgr2=0x7f41841f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.902+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f417000c4b0 tx=0x7f417000c970 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 shutdown_connections 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f416406bc00 0x7f416406e0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f418415fec0 0x7f418413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41841456d0 0x7f41841f6ec0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 --2- 172.21.15.62:0/3910568943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4184072ab0 0x7f41840b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 >> 172.21.15.62:0/3910568943 conn(0x7f41840b1730 msgr2=0x7f41840cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 shutdown_connections 2024-03-28T18:00:04.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:04.903+0000 7f4189c00640 1 -- 172.21.15.62:0/3910568943 wait complete. 2024-03-28T18:00:04.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T18:00:05.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.132+0000 7f5ad7e00640 1 Processor -- start 2024-03-28T18:00:05.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.132+0000 7f5ad7e00640 1 -- start start 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad7e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ad8072ab0 0x7f5ad8166cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad80cebb0 con 0x7f5ad81456d0 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad7e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad80cee90 con 0x7f5ad8160090 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad7e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad80cf170 con 0x7f5ad8072ab0 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad6a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5ad81456d0 0x7f5ad815f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52240/0 (socket says 172.21.15.62:52240) 2024-03-28T18:00:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad6a00640 1 -- 172.21.15.62:0/284701204 learned_addr learned my addr 172.21.15.62:0/284701204 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.134+0000 7f5ad7400640 1 --2- 172.21.15.62:0/284701204 >> v2:172.21.15.81:3300/0 conn(0x7f5ad8072ab0 0x7f5ad8166cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259687161 0 0) 0x7f5ad80cebb0 con 0x7f5ad81456d0 2024-03-28T18:00:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5abc003520 con 0x7f5ad81456d0 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288366053 0 0) 0x7f5abc003520 con 0x7f5ad81456d0 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 >> v2:172.21.15.81:3300/0 conn(0x7f5ad8072ab0 msgr2=0x7f5ad8166cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 --2- 172.21.15.62:0/284701204 >> v2:172.21.15.81:3300/0 conn(0x7f5ad8072ab0 0x7f5ad8166cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 >> v1:172.21.15.81:6789/0 conn(0x7f5ad8160090 legacy=0x7f5ad8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80cf450 con 0x7f5ad81456d0 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f5ac8001670 con 0x7f5ad81456d0 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 >> v1:172.21.15.62:6789/0 conn(0x7f5ad81456d0 legacy=0x7f5ad815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 0x7f5abc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5abc004800 0x7f5abc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc005210 0x7f5abc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80cf170 con 0x7f5abc003a00 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc00c160 con 0x7f5abc005210 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc00c360 con 0x7f5abc004800 2024-03-28T18:00:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.135+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 0x7f5abc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad7400640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc005210 0x7f5abc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6000640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5abc004800 0x7f5abc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6a00640 1 -- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5abc004800 msgr2=0x7f5abc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5abc004800 0x7f5abc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6a00640 1 -- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc005210 msgr2=0x7f5abc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc005210 0x7f5abc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6a00640 1 -- 172.21.15.62:0/284701204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc003520 con 0x7f5abc003a00 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad7400640 1 -- 172.21.15.62:0/284701204 reap_dead start 2024-03-28T18:00:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.136+0000 7f5ad6000640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5abc004800 0x7f5abc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.137+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 0x7f5abc003e20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5abc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9664944ff0fa34ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.137+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8007c20 con 0x7f5abc003a00 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.137+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8030070 con 0x7f5abc003a00 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.137+0000 7f5ad5600640 1 -- 172.21.15.62:0/284701204 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8005d30 con 0x7f5abc003a00 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 -- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 msgr2=0x7f5abc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 0x7f5abc003e20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5abc005830 comp rx=0 tx=0).stop 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 -- 172.21.15.62:0/284701204 shutdown_connections 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/284701204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5abc003a00 0x7f5abc003e20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 -- 172.21.15.62:0/284701204 >> 172.21.15.62:0/284701204 conn(0x7f5ad80b1730 msgr2=0x7f5ad80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 -- 172.21.15.62:0/284701204 shutdown_connections 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.138+0000 7f5ad7e00640 1 -- 172.21.15.62:0/284701204 wait complete. 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.140+0000 7f5ad7e00640 1 Processor -- start 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.140+0000 7f5ad7e00640 1 -- start start 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc003a00 0x7f5ad8138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ad81e5380 0x7f5ad81e5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815f980 con 0x7f5ad815da60 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80cf730 con 0x7f5abc003a00 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815fb00 con 0x7f5ad81e5380 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48730/0 (socket says 172.21.15.62:48730) 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 -- 172.21.15.62:0/1001851034 learned_addr learned my addr 172.21.15.62:0/1001851034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc003a00 0x7f5ad8138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad7400640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ad81e5380 0x7f5ad81e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 -- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ad81e5380 msgr2=0x7f5ad81e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ad81e5380 0x7f5ad81e5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 -- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc003a00 msgr2=0x7f5ad8138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.141+0000 7f5ad6000640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc003a00 0x7f5ad8138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5ad6000640 1 -- 172.21.15.62:0/1001851034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad81e6450 con 0x7f5ad815da60 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5ad6000640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d4e0 tx=0x7f5ac400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4005480 con 0x7f5ad815da60 2024-03-28T18:00:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac4017070 con 0x7f5ad815da60 2024-03-28T18:00:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40038a0 con 0x7f5ad815da60 2024-03-28T18:00:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad81e6730 con 0x7f5ad815da60 2024-03-28T18:00:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.142+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad81389b0 con 0x7f5ad815da60 2024-03-28T18:00:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.143+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad8073490 con 0x7f5ad815da60 2024-03-28T18:00:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.144+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5ac4003a40 con 0x7f5ad815da60 2024-03-28T18:00:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.145+0000 7f5acea00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 0x7f5ab806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.145+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5ac408a800 con 0x7f5ad815da60 2024-03-28T18:00:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.146+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 0x7f5ab806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.147+0000 7f5ad6a00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 0x7f5ab806e090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8034000 tx=0x7f5ac8039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:05.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.150+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5ac40593c0 con 0x7f5ad815da60 2024-03-28T18:00:05.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.394+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5ad804f4e0 con 0x7f5ad815da60 2024-03-28T18:00:05.395 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-03-28T18:00:05.395+0000 7f8492000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T18:00:05.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.399+0000 7f5acea00640 1 -- 172.21.15.62:0/1001851034 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5ac4058cc0 con 0x7f5ad815da60 2024-03-28T18:00:05.399 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 msgr2=0x7f5ab806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 0x7f5ab806e090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8034000 tx=0x7f5ac8039040 comp rx=0 tx=0).stop 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 msgr2=0x7f5ad815dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d4e0 tx=0x7f5ac400d9a0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 shutdown_connections 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5ab806bbd0 0x7f5ab806e090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5ad81e5380 0x7f5ad81e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ad815da60 0x7f5ad815dee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 --2- 172.21.15.62:0/1001851034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5abc003a00 0x7f5ad8138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:05.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 >> 172.21.15.62:0/1001851034 conn(0x7f5ad80b1730 msgr2=0x7f5ad8070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:05.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 shutdown_connections 2024-03-28T18:00:05.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.402+0000 7f5ad7e00640 1 -- 172.21.15.62:0/1001851034 wait complete. 2024-03-28T18:00:05.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T18:00:05.433 INFO:tasks.ceph.mds.d.smithi081.stderr: -4298> 2024-03-28T17:58:59.518+0000 7fdba1e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T18:00:05.503 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-28T18:00:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.645+0000 7f7ac0e00640 1 Processor -- start 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.645+0000 7f7ac0e00640 1 -- start start 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ac0e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7abc072ad0 0x7f7abc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ac0e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0b3470 con 0x7f7abc15e030 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ac0e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc1626d0 con 0x7f7abc0723f0 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ac0e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc1629b0 con 0x7f7abc072ad0 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ab9e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7abc15e030 0x7f7abc161a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52244/0 (socket says 172.21.15.62:52244) 2024-03-28T18:00:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ab9e00640 1 -- 172.21.15.62:0/2562838636 learned_addr learned my addr 172.21.15.62:0/2562838636 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.647+0000 7f7ab1400640 1 --2- 172.21.15.62:0/2562838636 >> v2:172.21.15.62:3300/0 conn(0x7f7abc072ad0 0x7f7abc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.648+0000 7f7ab1400640 1 -- 172.21.15.62:0/2562838636 >> v1:172.21.15.81:6789/0 conn(0x7f7abc0723f0 legacy=0x7f7abc0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.648+0000 7f7ab1400640 1 -- 172.21.15.62:0/2562838636 >> v1:172.21.15.62:6789/0 conn(0x7f7abc15e030 legacy=0x7f7abc161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.648+0000 7f7ab1400640 1 -- 172.21.15.62:0/2562838636 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc06cf30 con 0x7f7abc072ad0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.648+0000 7f7ab9400640 1 -- 172.21.15.62:0/2562838636 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665749724 0 0) 0x7f7abc0b3470 con 0x7f7abc15e030 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.649+0000 7f7ab1400640 1 --2- 172.21.15.62:0/2562838636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc072ad0 0x7f7abc070b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c4e14ae835ef85 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.649+0000 7f7ab9400640 1 -- 172.21.15.62:0/2562838636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4035020 con 0x7f7abc072ad0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.649+0000 7f7ab9400640 1 -- 172.21.15.62:0/2562838636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4031830 con 0x7f7abc072ad0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.649+0000 7f7ab9400640 1 -- 172.21.15.62:0/2562838636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4031c00 con 0x7f7abc072ad0 2024-03-28T18:00:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 -- 172.21.15.62:0/2562838636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc072ad0 msgr2=0x7f7abc070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/2562838636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc072ad0 0x7f7abc070b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402b8d0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 -- 172.21.15.62:0/2562838636 shutdown_connections 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/2562838636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc072ad0 0x7f7abc070b80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 -- 172.21.15.62:0/2562838636 >> 172.21.15.62:0/2562838636 conn(0x7f7abc0b1650 msgr2=0x7f7abc0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 -- 172.21.15.62:0/2562838636 shutdown_connections 2024-03-28T18:00:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.650+0000 7f7ac0e00640 1 -- 172.21.15.62:0/2562838636 wait complete. 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.652+0000 7f7ac0e00640 1 Processor -- start 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.652+0000 7f7ac0e00640 1 -- start start 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7abc0723f0 0x7f7abc15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7abc072ad0 0x7f7abc160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1623e0 con 0x7f7abc15e030 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0b37a0 con 0x7f7abc0723f0 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7ac0e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15f300 con 0x7f7abc072ad0 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7aba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7aba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48754/0 (socket says 172.21.15.62:48754) 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.653+0000 7f7aba800640 1 -- 172.21.15.62:0/46908601 learned_addr learned my addr 172.21.15.62:0/46908601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7ab9e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7abc0723f0 0x7f7abc15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7ab1400640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7abc072ad0 0x7f7abc160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 -- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7abc072ad0 msgr2=0x7f7abc160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7abc072ad0 0x7f7abc160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 -- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7abc0723f0 msgr2=0x7f7abc15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7abc0723f0 0x7f7abc15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 -- 172.21.15.62:0/46908601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc15b730 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7aba800640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b970 tx=0x7f7ab400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4019070 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab400cb40 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.654+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4014440 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.655+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc15ba10 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.655+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc15bfd0 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.656+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc0680c0 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.657+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7ab401c070 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.658+0000 7f7ab2a00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 0x7f7a9806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.658+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f7ab408a7f0 con 0x7f7abc15e030 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.659+0000 7f7ab9e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 0x7f7a9806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.660+0000 7f7ab9e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 0x7f7a9806e090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f60 tx=0x7f7aac007b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.663+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f7ab4059170 con 0x7f7abc15e030 2024-03-28T18:00:06.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:05.906+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7abc161ed0 con 0x7f7abc15e030 2024-03-28T18:00:06.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.450+0000 7f7ab2a00640 1 -- 172.21.15.62:0/46908601 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ab4058a70 con 0x7f7abc15e030 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.453+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 msgr2=0x7f7a9806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.453+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 0x7f7a9806e090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f60 tx=0x7f7aac007b70 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.453+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 msgr2=0x7f7abc1f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.453+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b970 tx=0x7f7ab400be30 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 shutdown_connections 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f7a9806bbd0 0x7f7a9806e090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc15e030 0x7f7abc1f0b00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f7abc072ad0 0x7f7abc160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 --2- 172.21.15.62:0/46908601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7abc0723f0 0x7f7abc15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 >> 172.21.15.62:0/46908601 conn(0x7f7abc0b1650 msgr2=0x7f7abc0ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 shutdown_connections 2024-03-28T18:00:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.454+0000 7f7ac0e00640 1 -- 172.21.15.62:0/46908601 wait complete. 2024-03-28T18:00:06.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T18:00:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.708+0000 7f03b4200640 1 Processor -- start 2024-03-28T18:00:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.708+0000 7f03b4200640 1 -- start start 2024-03-28T18:00:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b4200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f03ac0aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b4200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f03ac072ad0 0x7f03ac15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b4200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac15d1b0 con 0x7f03ac072690 2024-03-28T18:00:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b4200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0af040 con 0x7f03ac072ad0 2024-03-28T18:00:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b4200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac070d50 con 0x7f03ac0ae720 2024-03-28T18:00:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b0e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f03ac072690 0x7f03ac15f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52260/0 (socket says 172.21.15.62:52260) 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.710+0000 7f03b0e00640 1 -- 172.21.15.62:0/3850447819 learned_addr learned my addr 172.21.15.62:0/3850447819 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03b2200640 1 --2- 172.21.15.62:0/3850447819 >> v2:172.21.15.81:3300/0 conn(0x7f03ac072ad0 0x7f03ac15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03b1800640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f03ac0aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855230164 0 0) 0x7f03ac15d1b0 con 0x7f03ac072690 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0394003520 con 0x7f03ac072690 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88146305 0 0) 0x7f0394003520 con 0x7f03ac072690 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 msgr2=0x7f03ac0aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f03ac0aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 >> v2:172.21.15.81:3300/0 conn(0x7f03ac072ad0 msgr2=0x7f03ac15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 --2- 172.21.15.62:0/3850447819 >> v2:172.21.15.81:3300/0 conn(0x7f03ac072ad0 0x7f03ac15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac071030 con 0x7f03ac072690 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f0398001670 con 0x7f03ac072690 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 >> v1:172.21.15.62:6789/0 conn(0x7f03ac072690 legacy=0x7f03ac15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f0394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f0394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0394004d40 0x7f03940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0af040 con 0x7f03940039b0 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.711+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac070d50 con 0x7f0394004d40 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039400ba70 con 0x7f03ac0ae720 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f0394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b0e00640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f0394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b2200640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0394004d40 0x7f03940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 -- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 msgr2=0x7f0394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac0ae720 0x7f0394004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 -- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0394004d40 msgr2=0x7f03940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0394004d40 0x7f03940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.712+0000 7f03b1800640 1 -- 172.21.15.62:0/3850447819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394003520 con 0x7f03940039b0 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.714+0000 7f03b1800640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f0394003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f03a000ba60 tx=0x7f0394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725d61ccd80be9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.714+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0036020 con 0x7f03940039b0 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.714+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a0032630 con 0x7f03940039b0 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.714+0000 7f03abe00640 1 -- 172.21.15.62:0/3850447819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000b600 con 0x7f03940039b0 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 -- 172.21.15.62:0/3850447819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 msgr2=0x7f0394003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 --2- 172.21.15.62:0/3850447819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f0394003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f03a000ba60 tx=0x7f0394005830 comp rx=0 tx=0).stop 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b2200640 1 -- 172.21.15.62:0/3850447819 reap_dead start 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 -- 172.21.15.62:0/3850447819 shutdown_connections 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 -- 172.21.15.62:0/3850447819 >> 172.21.15.62:0/3850447819 conn(0x7f03ac0b1650 msgr2=0x7f03ac0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 -- 172.21.15.62:0/3850447819 shutdown_connections 2024-03-28T18:00:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.715+0000 7f03b4200640 1 -- 172.21.15.62:0/3850447819 wait complete. 2024-03-28T18:00:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.717+0000 7f03b4200640 1 Processor -- start 2024-03-28T18:00:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.717+0000 7f03b4200640 1 -- start start 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 0x7f03ac15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ac154180 0x7f03ac154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0718d0 con 0x7f03940039b0 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0715f0 con 0x7f03ac154180 2024-03-28T18:00:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b4200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac070a40 con 0x7f03ac15e200 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.718+0000 7f03b1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48772/0 (socket says 172.21.15.62:48772) 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 -- 172.21.15.62:0/1004813124 learned_addr learned my addr 172.21.15.62:0/1004813124 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b2200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ac154180 0x7f03ac154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b0e00640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 0x7f03ac15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 -- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 msgr2=0x7f03ac15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 0x7f03ac15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 -- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ac154180 msgr2=0x7f03ac154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ac154180 0x7f03ac154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 -- 172.21.15.62:0/1004813124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac154f40 con 0x7f03940039b0 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b0e00640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 0x7f03ac15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.719+0000 7f03b1800640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f03a000bb80 tx=0x7f03a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.720+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0041070 con 0x7f03940039b0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.720+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a00309c0 con 0x7f03940039b0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.720+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0036040 con 0x7f03940039b0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.720+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac155220 con 0x7f03940039b0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.720+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac1557e0 con 0x7f03940039b0 2024-03-28T18:00:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.721+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac15ebc0 con 0x7f03940039b0 2024-03-28T18:00:06.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.722+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f03a004c060 con 0x7f03940039b0 2024-03-28T18:00:06.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.723+0000 7f03a9600640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 0x7f038c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:06.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.723+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f03a00acd20 con 0x7f03940039b0 2024-03-28T18:00:06.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.723+0000 7f03b0e00640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 0x7f038c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.725+0000 7f03b0e00640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 0x7f038c06e090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0398001860 tx=0x7f0398000fe0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:06.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.727+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f03a007b610 con 0x7f03940039b0 2024-03-28T18:00:06.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:06.929+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03ac0680c0 con 0x7f03940039b0 2024-03-28T18:00:07.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.453+0000 7f03a9600640 1 -- 172.21.15.62:0/1004813124 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03a007af10 con 0x7f03940039b0 2024-03-28T18:00:07.454 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-03-28T18:00:07.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.457+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 msgr2=0x7f038c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.457+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 0x7f038c06e090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0398001860 tx=0x7f0398000fe0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.457+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 msgr2=0x7f03ac1f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.457+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f03a000bb80 tx=0x7f03a0004730 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 shutdown_connections 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f038c06bbd0 0x7f038c06e090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ac154180 0x7f03ac154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f03ac15e200 0x7f03ac15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 --2- 172.21.15.62:0/1004813124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03940039b0 0x7f03ac1f3980 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.458+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 >> 172.21.15.62:0/1004813124 conn(0x7f03ac0b1650 msgr2=0x7f03ac15f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.459+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 shutdown_connections 2024-03-28T18:00:07.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.459+0000 7f03b4200640 1 -- 172.21.15.62:0/1004813124 wait complete. 2024-03-28T18:00:07.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T18:00:07.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.704+0000 7f3c7f000640 1 Processor -- start 2024-03-28T18:00:07.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.704+0000 7f3c7f000640 1 -- start start 2024-03-28T18:00:07.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.705+0000 7f3c7f000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7815af20 0x7f3c780aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.705+0000 7f3c7f000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c7806ca60 0x7f3c78071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.705+0000 7f3c7f000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780b3330 con 0x7f3c78070a40 2024-03-28T18:00:07.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.705+0000 7f3c7f000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7806e980 con 0x7f3c7806ca60 2024-03-28T18:00:07.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.705+0000 7f3c7f000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780b3ee0 con 0x7f3c7815af20 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.706+0000 7f3c7d000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c7806ca60 0x7f3c78071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.706+0000 7f3c77e00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3c78070a40 0x7f3c780b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:56312/0 (socket says 172.21.15.62:56312) 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.706+0000 7f3c77e00640 1 -- 172.21.15.62:0/3143895415 learned_addr learned my addr 172.21.15.62:0/3143895415 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.706+0000 7f3c77400640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7815af20 0x7f3c780aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.707+0000 7f3c7d000640 1 -- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7815af20 msgr2=0x7f3c780aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.707+0000 7f3c7d000640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7815af20 0x7f3c780aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.707+0000 7f3c7d000640 1 -- 172.21.15.62:0/3143895415 >> v1:172.21.15.81:6789/0 conn(0x7f3c78070a40 legacy=0x7f3c780b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.707+0000 7f3c7d000640 1 -- 172.21.15.62:0/3143895415 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78072370 con 0x7f3c7806ca60 2024-03-28T18:00:07.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.708+0000 7f3c7d000640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7806ca60 0x7f3c78071e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c700097a0 tx=0x7f3c7002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69787c542474a0b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:07.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.709+0000 7f3c76a00640 1 -- 172.21.15.62:0/3143895415 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70031780 con 0x7f3c7806ca60 2024-03-28T18:00:07.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.709+0000 7f3c76a00640 1 -- 172.21.15.62:0/3143895415 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c70037070 con 0x7f3c7806ca60 2024-03-28T18:00:07.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.709+0000 7f3c76a00640 1 -- 172.21.15.62:0/3143895415 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70031b40 con 0x7f3c7806ca60 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.709+0000 7f3c7f000640 1 -- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7806ca60 msgr2=0x7f3c78071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.709+0000 7f3c7f000640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7806ca60 0x7f3c78071e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c700097a0 tx=0x7f3c7002b820 comp rx=0 tx=0).stop 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 -- 172.21.15.62:0/3143895415 shutdown_connections 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7806ca60 0x7f3c78071e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 --2- 172.21.15.62:0/3143895415 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7815af20 0x7f3c780aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 -- 172.21.15.62:0/3143895415 >> 172.21.15.62:0/3143895415 conn(0x7f3c780b1650 msgr2=0x7f3c780b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 -- 172.21.15.62:0/3143895415 shutdown_connections 2024-03-28T18:00:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.710+0000 7f3c7f000640 1 -- 172.21.15.62:0/3143895415 wait complete. 2024-03-28T18:00:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.712+0000 7f3c7f000640 1 Processor -- start 2024-03-28T18:00:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.712+0000 7f3c7f000640 1 -- start start 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7806ca60 0x7f3c7813c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c78070a40 0x7f3c781f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780b3890 con 0x7f3c7815af20 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780b35b0 con 0x7f3c78070a40 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7f000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780b3b70 con 0x7f3c7806ca60 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48780/0 (socket says 172.21.15.62:48780) 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c7d000640 1 -- 172.21.15.62:0/4232318157 learned_addr learned my addr 172.21.15.62:0/4232318157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c77400640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c78070a40 0x7f3c781f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.713+0000 7f3c77e00640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7806ca60 0x7f3c7813c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 -- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7806ca60 msgr2=0x7f3c7813c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7806ca60 0x7f3c7813c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 -- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c78070a40 msgr2=0x7f3c781f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:07.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c78070a40 0x7f3c781f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:07.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 -- 172.21.15.62:0/4232318157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c781f3db0 con 0x7f3c7815af20 2024-03-28T18:00:07.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7d000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c700097a0 tx=0x7f3c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70032610 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c700327b0 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7813ee80 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70032b80 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.714+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7813f440 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.716+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c780680c0 con 0x7f3c7815af20 2024-03-28T18:00:07.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.717+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3c7004c060 con 0x7f3c7815af20 2024-03-28T18:00:07.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.718+0000 7f3c6fe00640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 0x7f3c5406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:07.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.718+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f3c700ac440 con 0x7f3c7815af20 2024-03-28T18:00:07.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.718+0000 7f3c77e00640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 0x7f3c5406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:07.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.719+0000 7f3c77e00640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 0x7f3c5406e090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3c680012e0 tx=0x7f3c68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:07.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.722+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f3c70040070 con 0x7f3c7815af20 2024-03-28T18:00:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:07.926+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3c78157480 con 0x7f3c7815af20 2024-03-28T18:00:08.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.465+0000 7f3c6fe00640 1 -- 172.21.15.62:0/4232318157 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3c7002f050 con 0x7f3c7815af20 2024-03-28T18:00:08.466 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-03-28T18:00:08.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.469+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 msgr2=0x7f3c5406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.469+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 0x7f3c5406e090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3c680012e0 tx=0x7f3c68009040 comp rx=0 tx=0).stop 2024-03-28T18:00:08.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.469+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 msgr2=0x7f3c780713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.469+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c700097a0 tx=0x7f3c700046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 shutdown_connections 2024-03-28T18:00:08.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f3c5406bbd0 0x7f3c5406e090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c7815af20 0x7f3c780713e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c78070a40 0x7f3c781f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 --2- 172.21.15.62:0/4232318157 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f3c7806ca60 0x7f3c7813c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.470+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 >> 172.21.15.62:0/4232318157 conn(0x7f3c780b1650 msgr2=0x7f3c780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:08.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.471+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 shutdown_connections 2024-03-28T18:00:08.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.471+0000 7f3c7f000640 1 -- 172.21.15.62:0/4232318157 wait complete. 2024-03-28T18:00:08.501 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T18:00:08.501 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T18:00:08.501 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T18:00:08.502 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T18:00:08.502 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T18:00:08.502 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T18:00:08.502 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T18:00:08.503 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T18:00:08.503 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T18:00:08.503 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T18:00:08.503 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T18:00:08.503 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T18:00:08.503 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T18:00:08.504 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T18:00:08.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T18:00:08.504 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T18:00:08.522 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T18:00:08.523 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T18:00:08.524 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T18:00:08.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T18:00:08.525 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T18:00:08.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T18:00:08.527 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T18:00:08.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T18:00:08.569 INFO:tasks.ceph.mds.d:Started 2024-03-28T18:00:08.569 DEBUG:teuthology.parallel:result is None 2024-03-28T18:00:08.574 INFO:tasks.ceph.mds.a:Started 2024-03-28T18:00:08.574 DEBUG:teuthology.parallel:result is None 2024-03-28T18:00:08.575 INFO:tasks.ceph.mds.b:Started 2024-03-28T18:00:08.575 DEBUG:teuthology.parallel:result is None 2024-03-28T18:00:08.580 INFO:tasks.ceph.mds.c:Started 2024-03-28T18:00:08.580 DEBUG:teuthology.parallel:result is None 2024-03-28T18:00:08.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T18:00:08.726 INFO:tasks.ceph.mds.d.smithi081.stdout:starting mds.d at 2024-03-28T18:00:08.726 INFO:tasks.ceph.mds.b.smithi081.stdout:starting mds.b at 2024-03-28T18:00:08.732 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-03-28T18:00:08.732 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-03-28T18:00:08.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.774+0000 7fdce3400640 1 Processor -- start 2024-03-28T18:00:08.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.775+0000 7fdce3400640 1 -- start start 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce3400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce3400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0ad5c0 con 0x7fdcdc1624d0 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce3400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc072120 con 0x7fdcdc0719b0 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce3400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc072400 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce0a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdcdc1624d0 0x7fdcdc0b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52286/0 (socket says 172.21.15.62:52286) 2024-03-28T18:00:08.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdce0a00640 1 -- 172.21.15.62:0/4132118026 learned_addr learned my addr 172.21.15.62:0/4132118026 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:08.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.776+0000 7fdcdbe00640 1 --2- 172.21.15.62:0/4132118026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:08.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.777+0000 7fdcdb400640 1 -- 172.21.15.62:0/4132118026 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643695724 0 0) 0x7fdcdc0ad5c0 con 0x7fdcdc1624d0 2024-03-28T18:00:08.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.777+0000 7fdcdbe00640 1 -- 172.21.15.62:0/4132118026 >> v1:172.21.15.81:6789/0 conn(0x7fdcdc0719b0 legacy=0x7fdcdc0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.777+0000 7fdcdbe00640 1 -- 172.21.15.62:0/4132118026 >> v1:172.21.15.62:6789/0 conn(0x7fdcdc1624d0 legacy=0x7fdcdc0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.777+0000 7fdcdbe00640 1 -- 172.21.15.62:0/4132118026 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0726e0 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.778+0000 7fdcdbe00640 1 --2- 172.21.15.62:0/4132118026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc071410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009750 tx=0x7fdcc802b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c815d0ef003c0fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:08.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.778+0000 7fdcdb400640 1 -- 172.21.15.62:0/4132118026 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8037070 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.778+0000 7fdcdb400640 1 -- 172.21.15.62:0/4132118026 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8030710 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.778+0000 7fdcdb400640 1 -- 172.21.15.62:0/4132118026 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8030ae0 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 -- 172.21.15.62:0/4132118026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 msgr2=0x7fdcdc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 --2- 172.21.15.62:0/4132118026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc071410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009750 tx=0x7fdcc802b730 comp rx=0 tx=0).stop 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 -- 172.21.15.62:0/4132118026 shutdown_connections 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 --2- 172.21.15.62:0/4132118026 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc071410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 -- 172.21.15.62:0/4132118026 >> 172.21.15.62:0/4132118026 conn(0x7fdcdc066fb0 msgr2=0x7fdcdc0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 -- 172.21.15.62:0/4132118026 shutdown_connections 2024-03-28T18:00:08.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.779+0000 7fdce3400640 1 -- 172.21.15.62:0/4132118026 wait complete. 2024-03-28T18:00:08.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.781+0000 7fdce3400640 1 Processor -- start 2024-03-28T18:00:08.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.781+0000 7fdce3400640 1 -- start start 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc09c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 0x7fdcdc14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06ccb0 con 0x7fdcdc0719b0 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc071e40 con 0x7fdcdc1624d0 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce3400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06ce30 con 0x7fdcdc0b33c0 2024-03-28T18:00:08.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48802/0 (socket says 172.21.15.62:48802) 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.782+0000 7fdce0a00640 1 -- 172.21.15.62:0/2302051241 learned_addr learned my addr 172.21.15.62:0/2302051241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdcdbe00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc09c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce1400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 0x7fdcdc14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 -- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 msgr2=0x7fdcdc09c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc09c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 -- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 msgr2=0x7fdcdc14bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 0x7fdcdc14bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 -- 172.21.15.62:0/2302051241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0b0c30 con 0x7fdcdc0719b0 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce1400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 0x7fdcdc14bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.783+0000 7fdce0a00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c4a0 tx=0x7fdcd000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:08.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.784+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0018070 con 0x7fdcdc0719b0 2024-03-28T18:00:08.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.784+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd00074e0 con 0x7fdcdc0719b0 2024-03-28T18:00:08.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.784+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0004a20 con 0x7fdcdc0719b0 2024-03-28T18:00:08.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.784+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc09da70 con 0x7fdcdc0719b0 2024-03-28T18:00:08.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.784+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc09ccb0 con 0x7fdcdc0719b0 2024-03-28T18:00:08.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.785+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc0680c0 con 0x7fdcdc0719b0 2024-03-28T18:00:08.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.786+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdcd001b070 con 0x7fdcdc0719b0 2024-03-28T18:00:08.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.787+0000 7fdcd8c00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 0x7fdcbc06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:08.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.789+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdcd0089b80 con 0x7fdcdc0719b0 2024-03-28T18:00:08.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.789+0000 7fdcdbe00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 0x7fdcbc06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:08.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.789+0000 7fdcdbe00640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 0x7fdcbc06e0e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8034000 tx=0x7fdcc8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:08.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:08.793+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fdcd0058300 con 0x7fdcdc0719b0 2024-03-28T18:00:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.027+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdcdc0674a0 con 0x7fdcdc0719b0 2024-03-28T18:00:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.494+0000 7fdcd8c00640 1 -- 172.21.15.62:0/2302051241 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdcd005c3a0 con 0x7fdcdc0719b0 2024-03-28T18:00:09.494 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 msgr2=0x7fdcbc06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 0x7fdcbc06e0e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8034000 tx=0x7fdcc8008030 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 msgr2=0x7fdcdc09be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c4a0 tx=0x7fdcd000c960 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 shutdown_connections 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fdcbc06bc20 0x7fdcbc06e0e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcdc1624d0 0x7fdcdc14bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fdcdc0b33c0 0x7fdcdc09c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 --2- 172.21.15.62:0/2302051241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcdc0719b0 0x7fdcdc09be80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.498+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 >> 172.21.15.62:0/2302051241 conn(0x7fdcdc066fb0 msgr2=0x7fdcdc15ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.499+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 shutdown_connections 2024-03-28T18:00:09.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.499+0000 7fdce3400640 1 -- 172.21.15.62:0/2302051241 wait complete. 2024-03-28T18:00:09.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T18:00:09.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.736+0000 7f63ef200640 1 Processor -- start 2024-03-28T18:00:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.737+0000 7f63ef200640 1 -- start start 2024-03-28T18:00:09.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.738+0000 7f63ef200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63f015dd60 0x7f63f00b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.739+0000 7f63ef200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b99f0 0x7f63f00b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.739+0000 7f63ef200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00b6e60 con 0x7f63f00b5c30 2024-03-28T18:00:09.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.739+0000 7f63ef200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f00b60a0 con 0x7f63f015dd60 2024-03-28T18:00:09.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.739+0000 7f63ef200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b6c30 con 0x7f63f00b99f0 2024-03-28T18:00:09.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.740+0000 7f63ede00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63f015dd60 0x7f63f00b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.740+0000 7f63ede00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63f015dd60 0x7f63f00b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34686/0 (socket says 172.21.15.62:34686) 2024-03-28T18:00:09.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.740+0000 7f63ede00640 1 -- 172.21.15.62:0/1484916097 learned_addr learned my addr 172.21.15.62:0/1484916097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:09.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.740+0000 7f63ed400640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b99f0 0x7f63f00b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.741+0000 7f63ede00640 1 -- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b99f0 msgr2=0x7f63f00b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:09.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.741+0000 7f63ede00640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b99f0 0x7f63f00b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.741+0000 7f63ede00640 1 -- 172.21.15.62:0/1484916097 >> v1:172.21.15.81:6789/0 conn(0x7f63f00b5c30 legacy=0x7f63f0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.741+0000 7f63ede00640 1 -- 172.21.15.62:0/1484916097 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00b7100 con 0x7f63f015dd60 2024-03-28T18:00:09.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.742+0000 7f63ede00640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f015dd60 0x7f63f00b94b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009870 tx=0x7f63e002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b7d656c7d58eac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63eca00640 1 -- 172.21.15.62:0/1484916097 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e002f840 con 0x7f63f015dd60 2024-03-28T18:00:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63eca00640 1 -- 172.21.15.62:0/1484916097 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e002f9e0 con 0x7f63f015dd60 2024-03-28T18:00:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63eca00640 1 -- 172.21.15.62:0/1484916097 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e002fdb0 con 0x7f63f015dd60 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 -- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f015dd60 msgr2=0x7f63f00b94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f015dd60 0x7f63f00b94b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009870 tx=0x7f63e002ba20 comp rx=0 tx=0).stop 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 -- 172.21.15.62:0/1484916097 shutdown_connections 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b99f0 0x7f63f00b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 --2- 172.21.15.62:0/1484916097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f015dd60 0x7f63f00b94b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.743+0000 7f63ef200640 1 -- 172.21.15.62:0/1484916097 >> 172.21.15.62:0/1484916097 conn(0x7f63f0071ae0 msgr2=0x7f63f0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.744+0000 7f63ef200640 1 -- 172.21.15.62:0/1484916097 shutdown_connections 2024-03-28T18:00:09.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.744+0000 7f63ef200640 1 -- 172.21.15.62:0/1484916097 wait complete. 2024-03-28T18:00:09.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.745+0000 7f63ef200640 1 Processor -- start 2024-03-28T18:00:09.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.746+0000 7f63ef200640 1 -- start start 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.746+0000 7f63ef200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b5c30 0x7f63f0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.746+0000 7f63ef200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f00b99f0 0x7f63f00c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ef200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b79a0 con 0x7f63f015dd60 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ef200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b76c0 con 0x7f63f00b99f0 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ef200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f015e140 con 0x7f63f00b5c30 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48804/0 (socket says 172.21.15.62:48804) 2024-03-28T18:00:09.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ee800640 1 -- 172.21.15.62:0/2947731146 learned_addr learned my addr 172.21.15.62:0/2947731146 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:09.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ede00640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b5c30 0x7f63f0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.747+0000 7f63ed400640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f00b99f0 0x7f63f00c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 -- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b5c30 msgr2=0x7f63f0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b5c30 0x7f63f0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 -- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f00b99f0 msgr2=0x7f63f00c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f00b99f0 0x7f63f00c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 -- 172.21.15.62:0/2947731146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00c9f50 con 0x7f63f015dd60 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ee800640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c710 tx=0x7f63e800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8016020 con 0x7f63f015dd60 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8006d50 con 0x7f63f015dd60 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8003bb0 con 0x7f63f015dd60 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f00ca230 con 0x7f63f015dd60 2024-03-28T18:00:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.748+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f0160030 con 0x7f63f015dd60 2024-03-28T18:00:09.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.750+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0072b90 con 0x7f63f015dd60 2024-03-28T18:00:09.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.751+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f63e800a850 con 0x7f63f015dd60 2024-03-28T18:00:09.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.751+0000 7f63e6000640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 0x7f63cc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:09.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.752+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63e808a2a0 con 0x7f63f015dd60 2024-03-28T18:00:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.752+0000 7f63ede00640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 0x7f63cc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.754+0000 7f63ede00640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 0x7f63cc06e090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63e0036000 tx=0x7f63e0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:09.757+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f63e8020020 con 0x7f63f015dd60 2024-03-28T18:00:10.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.039+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f63f0160200 con 0x7f63f015dd60 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.040+0000 7f63e6000640 1 -- 172.21.15.62:0/2947731146 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f63e800ce20 con 0x7f63f015dd60 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCbrwVmiN6lNRAASRu6FTHQ/HbZ9UMk5eRuGA==", 2024-03-28T18:00:10.040 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCcrwVmnTb+HBAA4yirM35LUHfjsGj6/1LvwQ==", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCcrwVmOwlWCxAAy4KMrmuYYqT2SfSiGD66Qg==", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCcrwVmVtGuLRAARp0kxBW1fLsrcY7d98TeRA==", 2024-03-28T18:00:10.042 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQChrwVmZUQhKRAANCc7Q6HOlBbK5mE3MdsUxA==", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCkrwVmieT7MxAAA4Jy/TUwA8b8Aw8T8jJWUA==", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCorwVmFCZuAhAAA8Ijw5rMdzAhLPPkGiX5Aw==", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCrrwVm8HOoDRAAOxX4XAx0/PCKL3kDqHufVw==", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCyrwVm1V2WGBAAZQvGgbJYWczcHwiMkLSRPA==", 2024-03-28T18:00:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQC1rwVm5hgEMxAACg9Yuu+HjS/BKeea9szDjA==", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQC5rwVm792+AxAARa5j8feGpeKwSbGop8RWqQ==", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQC8rwVm6gEeDRAAlC30ODXpPSrfSowRFQmFIA==", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-03-28T18:00:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCcrwVm9FlXOxAAfVJXlLgfSVrvwPjHrZ1V6Q==", 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.048 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCdrwVmAatNFhAAZk3tH45tdhLWLMq+ofHs5w==", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCdrwVmfZcWCxAAnEDmPwLZf3n/I2datkH3vA==", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-03-28T18:00:10.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCdrwVmJ35SIRAAPt9Sl3SquQDeZ0hxYWTrUA==", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-03-28T18:00:10.050 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCarwVmIZL/KBAAmaVMTNm3m+cgI0ksd6ObvQ==", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mds", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmPVdwABAAqyJEPT2wxnCdJx9J+kscbw==", 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mds" 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.051 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mgr", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmZmRwABAAj9pl38WD/lnCdg7zD6AGow==", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-osd", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmD21wABAA9+dfFCqqty6Cag5Ey/Jlpg==", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-osd" 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmRHVwABAATHJ5X1RU/Vt4Y8FOivW1pw==", 2024-03-28T18:00:10.052 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmzH1wABAAlFzcoubxpVVAaLIBoMtNCg==", 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rgw", 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQDFrwVmkoZwABAAkEkwvutvouUztGTm9FyBEA==", 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-28T18:00:10.053 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCbrwVmkaDuKRAAwxEKu9XTzWl7KntUs79Npg==", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQCbrwVmoaC4HRAAkXQCXYk32KN+luxT8Vq4Fw==", 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-03-28T18:00:10.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-03-28T18:00:10.055 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 msgr2=0x7f63cc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 0x7f63cc06e090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63e0036000 tx=0x7f63e0008030 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 msgr2=0x7f63f015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c710 tx=0x7f63e800cbd0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 shutdown_connections 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f63cc06bbd0 0x7f63cc06e090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63f015dd60 0x7f63f015f6a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f00b99f0 0x7f63f00c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 --2- 172.21.15.62:0/2947731146 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f63f00b5c30 0x7f63f0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 >> 172.21.15.62:0/2947731146 conn(0x7f63f0071ae0 msgr2=0x7f63f00b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 shutdown_connections 2024-03-28T18:00:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.043+0000 7f63ef200640 1 -- 172.21.15.62:0/2947731146 wait complete. 2024-03-28T18:00:10.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-28T18:00:10.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.285+0000 7f73cfa00640 1 Processor -- start 2024-03-28T18:00:10.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.285+0000 7f73cfa00640 1 -- start start 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cfa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73c815e850 0x7f73c815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cfa00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8071020 0x7f73c8071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cfa00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80b3700 con 0x7f73c8144740 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cfa00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c815f1f0 con 0x7f73c815e850 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cfa00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c80b39e0 con 0x7f73c8071020 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cd000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f73c8144740 0x7f73c80b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52298/0 (socket says 172.21.15.62:52298) 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cd000640 1 -- 172.21.15.62:0/4075635106 learned_addr learned my addr 172.21.15.62:0/4075635106 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73c7e00640 1 --2- 172.21.15.62:0/4075635106 >> v2:172.21.15.62:3300/0 conn(0x7f73c815e850 0x7f73c815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73cda00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8071020 0x7f73c8071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73c7400640 1 -- 172.21.15.62:0/4075635106 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152144682 0 0) 0x7f73c80b3700 con 0x7f73c8144740 2024-03-28T18:00:10.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.287+0000 7f73c7400640 1 -- 172.21.15.62:0/4075635106 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73ac003520 con 0x7f73c8144740 2024-03-28T18:00:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.288+0000 7f73c7e00640 1 -- 172.21.15.62:0/4075635106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8071020 msgr2=0x7f73c8071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.288+0000 7f73c7e00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8071020 0x7f73c8071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.288+0000 7f73c7e00640 1 -- 172.21.15.62:0/4075635106 >> v1:172.21.15.62:6789/0 conn(0x7f73c8144740 legacy=0x7f73c80b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.288+0000 7f73c7e00640 1 -- 172.21.15.62:0/4075635106 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80b3cc0 con 0x7f73c815e850 2024-03-28T18:00:10.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.289+0000 7f73c7e00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c815e850 0x7f73c815ecb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73b00097a0 tx=0x7f73b002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81408de565478351 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.289+0000 7f73c7400640 1 -- 172.21.15.62:0/4075635106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0035020 con 0x7f73c815e850 2024-03-28T18:00:10.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.289+0000 7f73c7400640 1 -- 172.21.15.62:0/4075635106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b0031810 con 0x7f73c815e850 2024-03-28T18:00:10.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.289+0000 7f73c7400640 1 -- 172.21.15.62:0/4075635106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0031bc0 con 0x7f73c815e850 2024-03-28T18:00:10.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 -- 172.21.15.62:0/4075635106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c815e850 msgr2=0x7f73c815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c815e850 0x7f73c815ecb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73b00097a0 tx=0x7f73b002b8b0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 -- 172.21.15.62:0/4075635106 shutdown_connections 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8071020 0x7f73c8071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 --2- 172.21.15.62:0/4075635106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c815e850 0x7f73c815ecb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 -- 172.21.15.62:0/4075635106 >> 172.21.15.62:0/4075635106 conn(0x7f73c8066fb0 msgr2=0x7f73c80673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 -- 172.21.15.62:0/4075635106 shutdown_connections 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.290+0000 7f73cfa00640 1 -- 172.21.15.62:0/4075635106 wait complete. 2024-03-28T18:00:10.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.291+0000 7f73cfa00640 1 Processor -- start 2024-03-28T18:00:10.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.292+0000 7f73cfa00640 1 -- start start 2024-03-28T18:00:10.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.292+0000 7f73cfa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cfa00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8144740 0x7f73c815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cfa00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c815e850 0x7f73c80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cfa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c80acf20 con 0x7f73c8071020 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cfa00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c80b33c0 con 0x7f73c815e850 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cfa00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c80ad0a0 con 0x7f73c8144740 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48820/0 (socket says 172.21.15.62:48820) 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 -- 172.21.15.62:0/325778504 learned_addr learned my addr 172.21.15.62:0/325778504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 -- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8144740 msgr2=0x7f73c815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cda00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c815e850 0x7f73c80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73c7e00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8144740 0x7f73c815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8144740 0x7f73c815a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 -- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c815e850 msgr2=0x7f73c80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c815e850 0x7f73c80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.293+0000 7f73cd000640 1 -- 172.21.15.62:0/325778504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c809de30 con 0x7f73c8071020 2024-03-28T18:00:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73cd000640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f73b800c430 tx=0x7f73b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8017070 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8005ce0 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b80072f0 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c80c9730 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.294+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c80c9cf0 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.295+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c80680c0 con 0x7f73c8071020 2024-03-28T18:00:10.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.296+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f73b801a070 con 0x7f73c8071020 2024-03-28T18:00:10.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.297+0000 7f73c4c00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 0x7f73a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.297+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f73b8089bf0 con 0x7f73c8071020 2024-03-28T18:00:10.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.298+0000 7f73c7e00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 0x7f73a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.299+0000 7f73c7e00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 0x7f73a406e090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73b0036000 tx=0x7f73b0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.302+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f73b8058a20 con 0x7f73c8071020 2024-03-28T18:00:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.588+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f73c8142f10 con 0x7f73c8071020 2024-03-28T18:00:10.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.594+0000 7f73c4c00640 1 -- 172.21.15.62:0/325778504 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f73b8058320 con 0x7f73c8071020 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 msgr2=0x7f73a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 0x7f73a406e090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73b0036000 tx=0x7f73b0008030 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 msgr2=0x7f73c80c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f73b800c430 tx=0x7f73b800c8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 shutdown_connections 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f73a406bbd0 0x7f73a406e090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c815e850 0x7f73c80b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f73c8144740 0x7f73c815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 --2- 172.21.15.62:0/325778504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c8071020 0x7f73c80c9030 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 >> 172.21.15.62:0/325778504 conn(0x7f73c8066fb0 msgr2=0x7f73c80ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:10.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.596+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 shutdown_connections 2024-03-28T18:00:10.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.597+0000 7f73cfa00640 1 -- 172.21.15.62:0/325778504 wait complete. 2024-03-28T18:00:10.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-28T18:00:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.841+0000 7f19e3e00640 1 Processor -- start 2024-03-28T18:00:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.841+0000 7f19e3e00640 1 -- start start 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19e414bb00 0x7f19e4072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 0x7f19e40b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e3e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40b4aa0 con 0x7f19e40b3c60 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e40b4d80 con 0x7f19e414bb00 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4070b70 con 0x7f19e414bee0 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19e414bb00 0x7f19e4072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19e414bb00 0x7f19e4072690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48828/0 (socket says 172.21.15.62:48828) 2024-03-28T18:00:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e2a00640 1 -- 172.21.15.62:0/1734059359 learned_addr learned my addr 172.21.15.62:0/1734059359 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19da000640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 0x7f19e40b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.843+0000 7f19e2a00640 1 -- 172.21.15.62:0/1734059359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 msgr2=0x7f19e40b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.844+0000 7f19e2a00640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 0x7f19e40b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.844+0000 7f19e2a00640 1 -- 172.21.15.62:0/1734059359 >> v1:172.21.15.81:6789/0 conn(0x7f19e40b3c60 legacy=0x7f19e406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.844+0000 7f19e2a00640 1 -- 172.21.15.62:0/1734059359 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4070e50 con 0x7f19e414bb00 2024-03-28T18:00:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.844+0000 7f19da000640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 0x7f19e40b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.845+0000 7f19e2a00640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e4072690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4ed4923b92d4b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.845+0000 7f19e2000640 1 -- 172.21.15.62:0/1734059359 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40317c0 con 0x7f19e414bb00 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.845+0000 7f19e2000640 1 -- 172.21.15.62:0/1734059359 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4037070 con 0x7f19e414bb00 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.845+0000 7f19e2000640 1 -- 172.21.15.62:0/1734059359 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4031b80 con 0x7f19e414bb00 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 -- 172.21.15.62:0/1734059359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 msgr2=0x7f19e4072690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e4072690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d402b860 comp rx=0 tx=0).stop 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 -- 172.21.15.62:0/1734059359 shutdown_connections 2024-03-28T18:00:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e414bee0 0x7f19e40b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 --2- 172.21.15.62:0/1734059359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e4072690 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 -- 172.21.15.62:0/1734059359 >> 172.21.15.62:0/1734059359 conn(0x7f19e40b1730 msgr2=0x7f19e40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 -- 172.21.15.62:0/1734059359 shutdown_connections 2024-03-28T18:00:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.846+0000 7f19e3e00640 1 -- 172.21.15.62:0/1734059359 wait complete. 2024-03-28T18:00:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.848+0000 7f19e3e00640 1 Processor -- start 2024-03-28T18:00:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.848+0000 7f19e3e00640 1 -- start start 2024-03-28T18:00:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e40b3c60 0x7f19e415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19e414bee0 0x7f19e406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40b46e0 con 0x7f19e414bb00 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4071130 con 0x7f19e414bee0 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40b4860 con 0x7f19e40b3c60 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19da000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19da000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48844/0 (socket says 172.21.15.62:48844) 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19da000640 1 -- 172.21.15.62:0/2852777320 learned_addr learned my addr 172.21.15.62:0/2852777320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e2a00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e40b3c60 0x7f19e415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.849+0000 7f19e3400640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19e414bee0 0x7f19e406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 -- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e40b3c60 msgr2=0x7f19e415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e40b3c60 0x7f19e415bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 -- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19e414bee0 msgr2=0x7f19e406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19e414bee0 0x7f19e406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 -- 172.21.15.62:0/2852777320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e41652e0 con 0x7f19e414bb00 2024-03-28T18:00:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19da000640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c7f0 tx=0x7f19cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc005b90 con 0x7f19e414bb00 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc005d30 con 0x7f19e414bb00 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc006270 con 0x7f19e414bb00 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.850+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4143850 con 0x7f19e414bb00 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.851+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e4143470 con 0x7f19e414bb00 2024-03-28T18:00:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.852+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4145a40 con 0x7f19e414bb00 2024-03-28T18:00:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.853+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f19cc006620 con 0x7f19e414bb00 2024-03-28T18:00:10.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.854+0000 7f19dbe00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 0x7f19c006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:10.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.854+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19cc08a980 con 0x7f19e414bb00 2024-03-28T18:00:10.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.855+0000 7f19e2a00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 0x7f19c006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:10.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.856+0000 7f19e2a00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 0x7f19c006e090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036000 tx=0x7f19d403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:10.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:10.859+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f19cc015020 con 0x7f19e414bb00 2024-03-28T18:00:11.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.137+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f19e4073490 con 0x7f19e414bb00 2024-03-28T18:00:11.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.142+0000 7f19dbe00640 1 -- 172.21.15.62:0/2852777320 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19cc0597b0 con 0x7f19e414bb00 2024-03-28T18:00:11.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 msgr2=0x7f19c006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 0x7f19c006e090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036000 tx=0x7f19d403d040 comp rx=0 tx=0).stop 2024-03-28T18:00:11.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 msgr2=0x7f19e406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c7f0 tx=0x7f19cc00ccb0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 shutdown_connections 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f19c006bbd0 0x7f19c006e090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19e414bee0 0x7f19e406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e414bb00 0x7f19e406dc20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 --2- 172.21.15.62:0/2852777320 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f19e40b3c60 0x7f19e415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 >> 172.21.15.62:0/2852777320 conn(0x7f19e40b1730 msgr2=0x7f19e40b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 shutdown_connections 2024-03-28T18:00:11.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.145+0000 7f19e3e00640 1 -- 172.21.15.62:0/2852777320 wait complete. 2024-03-28T18:00:11.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-28T18:00:11.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.383+0000 7f23b2000640 1 Processor -- start 2024-03-28T18:00:11.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.384+0000 7f23b2000640 1 -- start start 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.385+0000 7f23b2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23ac160090 0x7f23ac070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.385+0000 7f23b2000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23ac072ab0 0x7f23ac070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.385+0000 7f23b2000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac0cebb0 con 0x7f23ac1456d0 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.385+0000 7f23b2000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac0cee90 con 0x7f23ac160090 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.385+0000 7f23b2000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac0cf170 con 0x7f23ac072ab0 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23ac160090 0x7f23ac070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23ac160090 0x7f23ac070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48852/0 (socket says 172.21.15.62:48852) 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 -- 172.21.15.62:0/4008868477 learned_addr learned my addr 172.21.15.62:0/4008868477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:11.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23b1600640 1 --2- 172.21.15.62:0/4008868477 >> v2:172.21.15.81:3300/0 conn(0x7f23ac072ab0 0x7f23ac070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 -- 172.21.15.62:0/4008868477 >> v2:172.21.15.81:3300/0 conn(0x7f23ac072ab0 msgr2=0x7f23ac070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:11.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 --2- 172.21.15.62:0/4008868477 >> v2:172.21.15.81:3300/0 conn(0x7f23ac072ab0 0x7f23ac070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 -- 172.21.15.62:0/4008868477 >> v1:172.21.15.81:6789/0 conn(0x7f23ac1456d0 legacy=0x7f23ac15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:11.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.386+0000 7f23abe00640 1 -- 172.21.15.62:0/4008868477 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0cf450 con 0x7f23ac160090 2024-03-28T18:00:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.387+0000 7f23abe00640 1 --2- 172.21.15.62:0/4008868477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac160090 0x7f23ac070a40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0b27b0 tx=0x7f239802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ecac29dc85b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.388+0000 7f23ab400640 1 -- 172.21.15.62:0/4008868477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398037070 con 0x7f23ac160090 2024-03-28T18:00:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.388+0000 7f23ab400640 1 -- 172.21.15.62:0/4008868477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398004230 con 0x7f23ac160090 2024-03-28T18:00:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.388+0000 7f23ab400640 1 -- 172.21.15.62:0/4008868477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398003610 con 0x7f23ac160090 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.388+0000 7f23b2000640 1 -- 172.21.15.62:0/4008868477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac160090 msgr2=0x7f23ac070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.388+0000 7f23b2000640 1 --2- 172.21.15.62:0/4008868477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac160090 0x7f23ac070a40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0b27b0 tx=0x7f239802b860 comp rx=0 tx=0).stop 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 -- 172.21.15.62:0/4008868477 shutdown_connections 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 --2- 172.21.15.62:0/4008868477 >> v2:172.21.15.81:3300/0 conn(0x7f23ac072ab0 0x7f23ac070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 --2- 172.21.15.62:0/4008868477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac160090 0x7f23ac070a40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 -- 172.21.15.62:0/4008868477 >> 172.21.15.62:0/4008868477 conn(0x7f23ac0b1730 msgr2=0x7f23ac0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 -- 172.21.15.62:0/4008868477 shutdown_connections 2024-03-28T18:00:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.389+0000 7f23b2000640 1 -- 172.21.15.62:0/4008868477 wait complete. 2024-03-28T18:00:11.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.390+0000 7f23b2000640 1 Processor -- start 2024-03-28T18:00:11.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 -- start start 2024-03-28T18:00:11.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 0x7f23ac1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f23ac160090 0x7f23ac1435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac06dd90 con 0x7f23ac1456d0 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0cf730 con 0x7f23ac072ab0 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.391+0000 7f23b2000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac06df10 con 0x7f23ac160090 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48864/0 (socket says 172.21.15.62:48864) 2024-03-28T18:00:11.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 -- 172.21.15.62:0/4009389166 learned_addr learned my addr 172.21.15.62:0/4009389166 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 -- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f23ac160090 msgr2=0x7f23ac1435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f23ac160090 0x7f23ac1435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 -- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 msgr2=0x7f23ac1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23b0c00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 0x7f23ac1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 0x7f23ac1f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23abe00640 1 -- 172.21.15.62:0/4009389166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac140fe0 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.392+0000 7f23b0c00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 0x7f23ac1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23abe00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f239802b8a0 tx=0x7f23980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398037050 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398043070 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398006040 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac1412c0 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.393+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac1f3f40 con 0x7f23ac1456d0 2024-03-28T18:00:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.394+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac073490 con 0x7f23ac1456d0 2024-03-28T18:00:11.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.395+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f23980061e0 con 0x7f23ac1456d0 2024-03-28T18:00:11.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.396+0000 7f23a8c00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 0x7f238c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.397+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23980ac790 con 0x7f23ac1456d0 2024-03-28T18:00:11.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.397+0000 7f23b0c00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 0x7f238c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.398+0000 7f23b0c00640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 0x7f238c06e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23a0003eb0 tx=0x7f23a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.401+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f239807b780 con 0x7f23ac1456d0 2024-03-28T18:00:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.681+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f23ac06e6e0 con 0x7f23ac1456d0 2024-03-28T18:00:11.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.687+0000 7f23a8c00640 1 -- 172.21.15.62:0/4009389166 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f239807b080 con 0x7f23ac1456d0 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.689+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 msgr2=0x7f238c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.689+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 0x7f238c06e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23a0003eb0 tx=0x7f23a0009040 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.689+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 msgr2=0x7f23ac1405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.689+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f239802b8a0 tx=0x7f23980046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 shutdown_connections 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f238c06bbd0 0x7f238c06e090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f23ac160090 0x7f23ac1435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23ac1456d0 0x7f23ac1405f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 --2- 172.21.15.62:0/4009389166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac072ab0 0x7f23ac1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 >> 172.21.15.62:0/4009389166 conn(0x7f23ac0b1730 msgr2=0x7f23ac0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 shutdown_connections 2024-03-28T18:00:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.690+0000 7f23b2000640 1 -- 172.21.15.62:0/4009389166 wait complete. 2024-03-28T18:00:11.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-28T18:00:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.927+0000 7fc414400640 1 Processor -- start 2024-03-28T18:00:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.927+0000 7fc414400640 1 -- start start 2024-03-28T18:00:11.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc414400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc40c0674e0 0x7fc40c166cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc414400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c06dd50 con 0x7fc40c1458c0 2024-03-28T18:00:11.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc414400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c0cfce0 con 0x7fc40c0d1730 2024-03-28T18:00:11.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc414400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c0cffa0 con 0x7fc40c0674e0 2024-03-28T18:00:11.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc412400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc40c0674e0 0x7fc40c166cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc411a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc40c1458c0 0x7fc40c0cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52306/0 (socket says 172.21.15.62:52306) 2024-03-28T18:00:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.929+0000 7fc411a00640 1 -- 172.21.15.62:0/976609001 learned_addr learned my addr 172.21.15.62:0/976609001 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.930+0000 7fc412400640 1 -- 172.21.15.62:0/976609001 >> v1:172.21.15.81:6789/0 conn(0x7fc40c0d1730 legacy=0x7fc40c066dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.930+0000 7fc412400640 1 -- 172.21.15.62:0/976609001 >> v1:172.21.15.62:6789/0 conn(0x7fc40c1458c0 legacy=0x7fc40c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.930+0000 7fc412400640 1 -- 172.21.15.62:0/976609001 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0d0280 con 0x7fc40c0674e0 2024-03-28T18:00:11.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.931+0000 7fc412400640 1 --2- 172.21.15.62:0/976609001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0674e0 0x7fc40c166cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc4040097a0 tx=0x7fc40402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4530702249631d65 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.931+0000 7fc40be00640 1 -- 172.21.15.62:0/976609001 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040315a0 con 0x7fc40c0674e0 2024-03-28T18:00:11.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.931+0000 7fc40be00640 1 -- 172.21.15.62:0/976609001 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404037070 con 0x7fc40c0674e0 2024-03-28T18:00:11.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.931+0000 7fc40be00640 1 -- 172.21.15.62:0/976609001 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404031960 con 0x7fc40c0674e0 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 -- 172.21.15.62:0/976609001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0674e0 msgr2=0x7fc40c166cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 --2- 172.21.15.62:0/976609001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0674e0 0x7fc40c166cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc4040097a0 tx=0x7fc40402b840 comp rx=0 tx=0).stop 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 -- 172.21.15.62:0/976609001 shutdown_connections 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 --2- 172.21.15.62:0/976609001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0674e0 0x7fc40c166cc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 -- 172.21.15.62:0/976609001 >> 172.21.15.62:0/976609001 conn(0x7fc40c0d0ba0 msgr2=0x7fc40c0d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 -- 172.21.15.62:0/976609001 shutdown_connections 2024-03-28T18:00:11.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.932+0000 7fc414400640 1 -- 172.21.15.62:0/976609001 wait complete. 2024-03-28T18:00:11.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.934+0000 7fc414400640 1 Processor -- start 2024-03-28T18:00:11.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.934+0000 7fc414400640 1 -- start start 2024-03-28T18:00:11.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc40c0674e0 0x7fc40c1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc40c1458c0 0x7fc40c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06f3c0 con 0x7fc40c0d1730 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06f240 con 0x7fc40c1458c0 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc414400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06f540 con 0x7fc40c0674e0 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48878/0 (socket says 172.21.15.62:48878) 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 -- 172.21.15.62:0/602798709 learned_addr learned my addr 172.21.15.62:0/602798709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 -- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc40c0674e0 msgr2=0x7fc40c1f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc40c0674e0 0x7fc40c1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 -- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc40c1458c0 msgr2=0x7fc40c13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc40c1458c0 0x7fc40c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.935+0000 7fc411000640 1 -- 172.21.15.62:0/602798709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c13e900 con 0x7fc40c0d1730 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc411000640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c430 tx=0x7fc3f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8011790 con 0x7fc40c0d1730 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f8017070 con 0x7fc40c0d1730 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8011b70 con 0x7fc40c0d1730 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c13d860 con 0x7fc40c0d1730 2024-03-28T18:00:11.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.936+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c13de20 con 0x7fc40c0d1730 2024-03-28T18:00:11.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.937+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c0680c0 con 0x7fc40c0d1730 2024-03-28T18:00:11.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.938+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc3f8011d10 con 0x7fc40c0d1730 2024-03-28T18:00:11.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.939+0000 7fc409600640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 0x7fc3ec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:11.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.939+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3f808a650 con 0x7fc40c0d1730 2024-03-28T18:00:11.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.940+0000 7fc411a00640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 0x7fc3ec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:11.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.941+0000 7fc411a00640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 0x7fc3ec06e090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4000012e0 tx=0x7fc400001590 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:11.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:11.944+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fc3f80593f0 con 0x7fc40c0d1730 2024-03-28T18:00:12.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.219+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc40c072e70 con 0x7fc40c0d1730 2024-03-28T18:00:12.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.224+0000 7fc409600640 1 -- 172.21.15.62:0/602798709 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc3f8058cf0 con 0x7fc40c0d1730 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.226+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 msgr2=0x7fc3ec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.226+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 0x7fc3ec06e090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4000012e0 tx=0x7fc400001590 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.226+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 msgr2=0x7fc40c13d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c430 tx=0x7fc3f800c8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 shutdown_connections 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fc3ec06bbd0 0x7fc3ec06e090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc40c1458c0 0x7fc40c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc40c0d1730 0x7fc40c13d160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 --2- 172.21.15.62:0/602798709 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fc40c0674e0 0x7fc40c1f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 >> 172.21.15.62:0/602798709 conn(0x7fc40c0d0ba0 msgr2=0x7fc40c15d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:12.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 shutdown_connections 2024-03-28T18:00:12.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.227+0000 7fc414400640 1 -- 172.21.15.62:0/602798709 wait complete. 2024-03-28T18:00:12.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-28T18:00:12.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.472+0000 7f4e9b200640 1 Processor -- start 2024-03-28T18:00:12.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.473+0000 7f4e9b200640 1 -- start start 2024-03-28T18:00:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.474+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e9c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.474+0000 7f4e9b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c1458d0 con 0x7f4e9c1454f0 2024-03-28T18:00:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.474+0000 7f4e9b200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0b2b90 con 0x7f4e9c0cd100 2024-03-28T18:00:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.474+0000 7f4e9b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0cd8d0 con 0x7f4e9c06d100 2024-03-28T18:00:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.475+0000 7f4e99e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4e9c1454f0 0x7f4e9c06c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52316/0 (socket says 172.21.15.62:52316) 2024-03-28T18:00:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.475+0000 7f4e99e00640 1 -- 172.21.15.62:0/1101939395 learned_addr learned my addr 172.21.15.62:0/1101939395 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.475+0000 7f4e93e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e9c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.475+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169809870 0 0) 0x7f4e9c1458d0 con 0x7f4e9c1454f0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.475+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e80003520 con 0x7f4e9c1454f0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908333093 0 0) 0x7f4e80003520 con 0x7f4e9c1454f0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 msgr2=0x7f4e9c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e9c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 >> v1:172.21.15.81:6789/0 conn(0x7f4e9c0cd100 legacy=0x7f4e9c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0b2e70 con 0x7f4e9c1454f0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f4e8c001670 con 0x7f4e9c1454f0 2024-03-28T18:00:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 >> v1:172.21.15.62:6789/0 conn(0x7f4e9c1454f0 legacy=0x7f4e9c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e80004800 0x7f4e80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0cd8d0 con 0x7f4e800039e0 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8000baa0 con 0x7f4e80004800 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8000bca0 con 0x7f4e9c06d100 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.476+0000 7f4e99e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e93e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e80004800 0x7f4e80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e9a800640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e99e00640 1 -- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 msgr2=0x7f4e800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e99e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c06d100 0x7f4e800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e99e00640 1 -- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e80004800 msgr2=0x7f4e80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e99e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e80004800 0x7f4e80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.477+0000 7f4e99e00640 1 -- 172.21.15.62:0/1101939395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80003520 con 0x7f4e800039e0 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.478+0000 7f4e99e00640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e80003e00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e3b343768e19aa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.478+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c007d10 con 0x7f4e800039e0 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c030070 con 0x7f4e800039e0 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e99400640 1 -- 172.21.15.62:0/1101939395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c005cf0 con 0x7f4e800039e0 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 -- 172.21.15.62:0/1101939395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 msgr2=0x7f4e80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 --2- 172.21.15.62:0/1101939395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e80003e00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e80005830 comp rx=0 tx=0).stop 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9a800640 1 -- 172.21.15.62:0/1101939395 reap_dead start 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 -- 172.21.15.62:0/1101939395 shutdown_connections 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 -- 172.21.15.62:0/1101939395 >> 172.21.15.62:0/1101939395 conn(0x7f4e9c074340 msgr2=0x7f4e9c066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 -- 172.21.15.62:0/1101939395 shutdown_connections 2024-03-28T18:00:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.479+0000 7f4e9b200640 1 -- 172.21.15.62:0/1101939395 wait complete. 2024-03-28T18:00:12.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.481+0000 7f4e9b200640 1 Processor -- start 2024-03-28T18:00:12.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.481+0000 7f4e9b200640 1 -- start start 2024-03-28T18:00:12.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 0x7f4e9c13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 0x7f4e9c13cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c15c010 con 0x7f4e800039e0 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b34c0 con 0x7f4e9c142300 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e9b200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0cd570 con 0x7f4e9c142780 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e99e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e99e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48900/0 (socket says 172.21.15.62:48900) 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.482+0000 7f4e99e00640 1 -- 172.21.15.62:0/187184009 learned_addr learned my addr 172.21.15.62:0/187184009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e9a800640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 0x7f4e9c13cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e93e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 0x7f4e9c13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 -- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 msgr2=0x7f4e9c13cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 0x7f4e9c13cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 -- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 msgr2=0x7f4e9c13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 0x7f4e9c13c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 -- 172.21.15.62:0/187184009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c142d10 con 0x7f4e800039e0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e93e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 0x7f4e9c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e9a800640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 0x7f4e9c13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e99e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e9c06e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.483+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c007d10 con 0x7f4e800039e0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.484+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c004bf0 con 0x7f4e800039e0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.484+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c007070 con 0x7f4e800039e0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.484+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c142ff0 con 0x7f4e800039e0 2024-03-28T18:00:12.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.484+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c1435b0 con 0x7f4e800039e0 2024-03-28T18:00:12.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.485+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c04f400 con 0x7f4e800039e0 2024-03-28T18:00:12.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.486+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4e8c007210 con 0x7f4e800039e0 2024-03-28T18:00:12.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.487+0000 7f4e92000640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 0x7f4e7806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:12.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.487+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e8c0abf20 con 0x7f4e800039e0 2024-03-28T18:00:12.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.487+0000 7f4e93e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 0x7f4e7806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:12.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.488+0000 7f4e93e00640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 0x7f4e7806e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c1f5220 tx=0x7f4e84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.491+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f4e8c07acc0 con 0x7f4e800039e0 2024-03-28T18:00:12.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.769+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f4e9c067820 con 0x7f4e800039e0 2024-03-28T18:00:12.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.774+0000 7f4e92000640 1 -- 172.21.15.62:0/187184009 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e8c07a5c0 con 0x7f4e800039e0 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 msgr2=0x7f4e7806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 0x7f4e7806e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c1f5220 tx=0x7f4e84008040 comp rx=0 tx=0).stop 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 msgr2=0x7f4e9c0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e9c06e1d0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 shutdown_connections 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f4e7806bbd0 0x7f4e7806e090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f4e9c142780 0x7f4e9c13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e9c142300 0x7f4e9c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 --2- 172.21.15.62:0/187184009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e800039e0 0x7f4e9c0cb230 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.776+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 >> 172.21.15.62:0/187184009 conn(0x7f4e9c074340 msgr2=0x7f4e9c15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.777+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 shutdown_connections 2024-03-28T18:00:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:12.777+0000 7f4e9b200640 1 -- 172.21.15.62:0/187184009 wait complete. 2024-03-28T18:00:12.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-28T18:00:13.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.009+0000 7f66c6000640 1 Processor -- start 2024-03-28T18:00:13.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.009+0000 7f66c6000640 1 -- start start 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.010+0000 7f66c6000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66c015f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c6000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f66c0072ab0 0x7f66c015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c6000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00cebb0 con 0x7f66c006cd80 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c6000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00cee90 con 0x7f66c0072ab0 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c6000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00cf170 con 0x7f66c01456d0 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c5600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f66c006cd80 0x7f66c006d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52322/0 (socket says 172.21.15.62:52322) 2024-03-28T18:00:13.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c5600640 1 -- 172.21.15.62:0/2685381920 learned_addr learned my addr 172.21.15.62:0/2685381920 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:13.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66c4c00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66c015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2685381920 >> v2:172.21.15.81:3300/0 conn(0x7f66c0072ab0 0x7f66c015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.011+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209046127 0 0) 0x7f66c00cebb0 con 0x7f66c006cd80 2024-03-28T18:00:13.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a8003520 con 0x7f66c006cd80 2024-03-28T18:00:13.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1035114263 0 0) 0x7f66a8003520 con 0x7f66c006cd80 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 msgr2=0x7f66c015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66c015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 >> v2:172.21.15.81:3300/0 conn(0x7f66c0072ab0 msgr2=0x7f66c015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 --2- 172.21.15.62:0/2685381920 >> v2:172.21.15.81:3300/0 conn(0x7f66c0072ab0 0x7f66c015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00cf450 con 0x7f66c006cd80 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f66b80019f0 con 0x7f66c006cd80 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 >> v1:172.21.15.62:6789/0 conn(0x7f66c006cd80 legacy=0x7f66c006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66a8003f80 0x7f66a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00cee90 con 0x7f66a8003f80 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00cf170 con 0x7f66a8004d40 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a800ba70 con 0x7f66c01456d0 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.012+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66a8003f80 0x7f66a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66c5600640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66c4c00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66bfe00640 1 -- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 msgr2=0x7f66a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66c01456d0 0x7f66a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66bfe00640 1 -- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66a8004d40 msgr2=0x7f66a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.013+0000 7f66bfe00640 1 -- 172.21.15.62:0/2685381920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8003520 con 0x7f66a8003f80 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.014+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66a8003f80 0x7f66a8004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009a80 tx=0x7f66a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e4dbfd5abcc683 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.014+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac009680 con 0x7f66a8003f80 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.014+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac005990 con 0x7f66a8003f80 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.014+0000 7f66bf400640 1 -- 172.21.15.62:0/2685381920 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac005d80 con 0x7f66a8003f80 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 -- 172.21.15.62:0/2685381920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66a8003f80 msgr2=0x7f66a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 --2- 172.21.15.62:0/2685381920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66a8003f80 0x7f66a8004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009a80 tx=0x7f66a8005830 comp rx=0 tx=0).stop 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c5600640 1 -- 172.21.15.62:0/2685381920 reap_dead start 2024-03-28T18:00:13.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 -- 172.21.15.62:0/2685381920 shutdown_connections 2024-03-28T18:00:13.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 -- 172.21.15.62:0/2685381920 >> 172.21.15.62:0/2685381920 conn(0x7f66c00b1730 msgr2=0x7f66c00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:13.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 -- 172.21.15.62:0/2685381920 shutdown_connections 2024-03-28T18:00:13.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.015+0000 7f66c6000640 1 -- 172.21.15.62:0/2685381920 wait complete. 2024-03-28T18:00:13.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.017+0000 7f66c6000640 1 Processor -- start 2024-03-28T18:00:13.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.017+0000 7f66c6000640 1 -- start start 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66a8003f80 0x7f66c013ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66c015c2a0 0x7f66c015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c015f980 con 0x7f66c013bf60 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00cf730 con 0x7f66c015c2a0 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c6000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c015fb00 con 0x7f66a8003f80 2024-03-28T18:00:13.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48924/0 (socket says 172.21.15.62:48924) 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.018+0000 7f66c5600640 1 -- 172.21.15.62:0/2791390376 learned_addr learned my addr 172.21.15.62:0/2791390376 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c4c00640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66a8003f80 0x7f66c013ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66bfe00640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66c015c2a0 0x7f66c015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 -- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66a8003f80 msgr2=0x7f66c013ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66a8003f80 0x7f66c013ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 -- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66c015c2a0 msgr2=0x7f66c015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66c015c2a0 0x7f66c015c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 -- 172.21.15.62:0/2791390376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c013e330 con 0x7f66c013bf60 2024-03-28T18:00:13.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.019+0000 7f66c5600640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b27b0 tx=0x7f66b800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.020+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b800ec10 con 0x7f66c013bf60 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.020+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b800edb0 con 0x7f66c013bf60 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.020+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8014650 con 0x7f66c013bf60 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.020+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c015ce10 con 0x7f66c013bf60 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.020+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c015d3d0 con 0x7f66c013bf60 2024-03-28T18:00:13.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.021+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0073490 con 0x7f66c013bf60 2024-03-28T18:00:13.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.022+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f66b80147f0 con 0x7f66c013bf60 2024-03-28T18:00:13.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.023+0000 7f66bcc00640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 0x7f66a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.023+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66b808a5a0 con 0x7f66c013bf60 2024-03-28T18:00:13.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.023+0000 7f66c4c00640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 0x7f66a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.024+0000 7f66c4c00640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 0x7f66a006e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009930 tx=0x7f66b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.027+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f66b8059350 con 0x7f66c013bf60 2024-03-28T18:00:13.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.299+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f66c004f4e0 con 0x7f66c013bf60 2024-03-28T18:00:13.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.304+0000 7f66bcc00640 1 -- 172.21.15.62:0/2791390376 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f66b8058c50 con 0x7f66c013bf60 2024-03-28T18:00:13.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 msgr2=0x7f66a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 0x7f66a006e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009930 tx=0x7f66b4008040 comp rx=0 tx=0).stop 2024-03-28T18:00:13.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 msgr2=0x7f66c013c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b27b0 tx=0x7f66b800e530 comp rx=0 tx=0).stop 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 shutdown_connections 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f66a006bbd0 0x7f66a006e090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c013bf60 0x7f66c013c3e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66c015c2a0 0x7f66c015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 --2- 172.21.15.62:0/2791390376 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f66a8003f80 0x7f66c013ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 >> 172.21.15.62:0/2791390376 conn(0x7f66c00b1730 msgr2=0x7f66c0140fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.306+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 shutdown_connections 2024-03-28T18:00:13.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.307+0000 7f66c6000640 1 -- 172.21.15.62:0/2791390376 wait complete. 2024-03-28T18:00:13.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:13.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.538+0000 7f94dfe00640 1 Processor -- start 2024-03-28T18:00:13.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.538+0000 7f94dfe00640 1 -- start start 2024-03-28T18:00:13.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 0x7f94d80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dfe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f94d80cf3c0 0x7f94d80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dfe00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d80cf7d0 con 0x7f94d8152f90 2024-03-28T18:00:13.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dfe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d8160110 con 0x7f94d806ccb0 2024-03-28T18:00:13.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dfe00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d81603c0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dde00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f94d80cf3c0 0x7f94d80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 0x7f94d80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dde00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f94d80cf3c0 0x7f94d80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34790/0 (socket says 172.21.15.62:34790) 2024-03-28T18:00:13.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.540+0000 7f94dde00640 1 -- 172.21.15.62:0/3992203093 learned_addr learned my addr 172.21.15.62:0/3992203093 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:13.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.541+0000 7f94dde00640 1 -- 172.21.15.62:0/3992203093 >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 msgr2=0x7f94d80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.541+0000 7f94dde00640 1 --2- 172.21.15.62:0/3992203093 >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 0x7f94d80cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.541+0000 7f94dde00640 1 -- 172.21.15.62:0/3992203093 >> v1:172.21.15.81:6789/0 conn(0x7f94d8152f90 legacy=0x7f94d815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.541+0000 7f94dde00640 1 -- 172.21.15.62:0/3992203093 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d81606a0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.541+0000 7f94dca00640 1 --2- 172.21.15.62:0/3992203093 >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 0x7f94d80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:00:13.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.542+0000 7f94dde00640 1 --2- 172.21.15.62:0/3992203093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b44b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94d809e040 tx=0x7f94d002bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02905186979c478 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.543+0000 7f94d7e00640 1 -- 172.21.15.62:0/3992203093 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d002cb10 con 0x7f94d80cf3c0 2024-03-28T18:00:13.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.543+0000 7f94d7e00640 1 -- 172.21.15.62:0/3992203093 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d002ccb0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.543+0000 7f94d7e00640 1 -- 172.21.15.62:0/3992203093 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d002ce60 con 0x7f94d80cf3c0 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 -- 172.21.15.62:0/3992203093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d80cf3c0 msgr2=0x7f94d80b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 --2- 172.21.15.62:0/3992203093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b44b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94d809e040 tx=0x7f94d002bf70 comp rx=0 tx=0).stop 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 -- 172.21.15.62:0/3992203093 shutdown_connections 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 --2- 172.21.15.62:0/3992203093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b44b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 --2- 172.21.15.62:0/3992203093 >> v2:172.21.15.62:3300/0 conn(0x7f94d806ccb0 0x7f94d80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 -- 172.21.15.62:0/3992203093 >> 172.21.15.62:0/3992203093 conn(0x7f94d8066fb0 msgr2=0x7f94d80673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 -- 172.21.15.62:0/3992203093 shutdown_connections 2024-03-28T18:00:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.544+0000 7f94dfe00640 1 -- 172.21.15.62:0/3992203093 wait complete. 2024-03-28T18:00:13.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.546+0000 7f94dfe00640 1 Processor -- start 2024-03-28T18:00:13.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.546+0000 7f94dfe00640 1 -- start start 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94d806ccb0 0x7f94d81414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d8152f90 0x7f94d815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815c5d0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b3f90 con 0x7f94d8152f90 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dfe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815c750 con 0x7f94d806ccb0 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48930/0 (socket says 172.21.15.62:48930) 2024-03-28T18:00:13.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 -- 172.21.15.62:0/664497214 learned_addr learned my addr 172.21.15.62:0/664497214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dde00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d8152f90 0x7f94d815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 -- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94d806ccb0 msgr2=0x7f94d81414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dd400640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94d806ccb0 0x7f94d81414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94d806ccb0 0x7f94d81414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 -- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d8152f90 msgr2=0x7f94d815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d8152f90 0x7f94d815ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.547+0000 7f94dca00640 1 -- 172.21.15.62:0/664497214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8141a10 con 0x7f94d80cf3c0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94dca00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e960 tx=0x7f94c400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4017070 con 0x7f94d80cf3c0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c400ca00 con 0x7f94d80cf3c0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c400cdd0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d80af820 con 0x7f94d80cf3c0 2024-03-28T18:00:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.548+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d80afde0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.549+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d80680c0 con 0x7f94d80cf3c0 2024-03-28T18:00:13.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.550+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f94c401a070 con 0x7f94d80cf3c0 2024-03-28T18:00:13.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.551+0000 7f94d5600640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.551+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94c408a630 con 0x7f94d80cf3c0 2024-03-28T18:00:13.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.552+0000 7f94dd400640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.553+0000 7f94dd400640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 0x7f94b806e090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94d8072d90 tx=0x7f94cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.556+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f94c4059460 con 0x7f94d80cf3c0 2024-03-28T18:00:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.754+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94d80b3500 con 0x7f94d80cf3c0 2024-03-28T18:00:13.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.756+0000 7f94d5600640 1 -- 172.21.15.62:0/664497214 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12843 (secure 0 0 0) 0x7f94c4058d60 con 0x7f94d80cf3c0 2024-03-28T18:00:13.756 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:13.756 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T18:00:09.507039+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 msgr2=0x7f94b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 0x7f94b806e090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94d8072d90 tx=0x7f94cc009040 comp rx=0 tx=0).stop 2024-03-28T18:00:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 msgr2=0x7f94d80b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e960 tx=0x7f94c400ee20 comp rx=0 tx=0).stop 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 shutdown_connections 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94d8152f90 0x7f94d815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d80cf3c0 0x7f94d80b01c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 --2- 172.21.15.62:0/664497214 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f94d806ccb0 0x7f94d81414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 >> 172.21.15.62:0/664497214 conn(0x7f94d8066fb0 msgr2=0x7f94d80b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 shutdown_connections 2024-03-28T18:00:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.758+0000 7f94dfe00640 1 -- 172.21.15.62:0/664497214 wait complete. 2024-03-28T18:00:13.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:42.378514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:00:13.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:00:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.990+0000 7f5b86c00640 1 Processor -- start 2024-03-28T18:00:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.990+0000 7f5b86c00640 1 -- start start 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b86c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b8015e030 0x7f5b80161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b86c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 0x7f5b80161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b86c00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800b3470 con 0x7f5b800723f0 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b86c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b801629b0 con 0x7f5b8015e030 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b86c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800b25b0 con 0x7f5b80072ad0 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b7fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b8015e030 0x7f5b80161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b7fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b8015e030 0x7f5b80161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42472/0 (socket says 172.21.15.62:42472) 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.992+0000 7f5b7fe00640 1 -- 172.21.15.62:0/3723962464 learned_addr learned my addr 172.21.15.62:0/3723962464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7f400640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 0x7f5b80161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7fe00640 1 -- 172.21.15.62:0/3723962464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 msgr2=0x7f5b80161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7fe00640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 0x7f5b80161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7fe00640 1 -- 172.21.15.62:0/3723962464 >> v1:172.21.15.81:6789/0 conn(0x7f5b800723f0 legacy=0x7f5b800ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:13.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7fe00640 1 -- 172.21.15.62:0/3723962464 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b801626d0 con 0x7f5b8015e030 2024-03-28T18:00:13.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.993+0000 7f5b7f400640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 0x7f5b80161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:13.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.994+0000 7f5b7fe00640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b80161a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009870 tx=0x7f5b7002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61982c6b14e6a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:13.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.994+0000 7f5b7ea00640 1 -- 172.21.15.62:0/3723962464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70035020 con 0x7f5b8015e030 2024-03-28T18:00:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.994+0000 7f5b7ea00640 1 -- 172.21.15.62:0/3723962464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70031640 con 0x7f5b8015e030 2024-03-28T18:00:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.994+0000 7f5b7ea00640 1 -- 172.21.15.62:0/3723962464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7002d040 con 0x7f5b8015e030 2024-03-28T18:00:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 -- 172.21.15.62:0/3723962464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 msgr2=0x7f5b80161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b80161a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009870 tx=0x7f5b7002b8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 -- 172.21.15.62:0/3723962464 shutdown_connections 2024-03-28T18:00:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b80072ad0 0x7f5b80161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 --2- 172.21.15.62:0/3723962464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b80161a00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 -- 172.21.15.62:0/3723962464 >> 172.21.15.62:0/3723962464 conn(0x7f5b800b1650 msgr2=0x7f5b800b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 -- 172.21.15.62:0/3723962464 shutdown_connections 2024-03-28T18:00:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.995+0000 7f5b86c00640 1 -- 172.21.15.62:0/3723962464 wait complete. 2024-03-28T18:00:13.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.997+0000 7f5b86c00640 1 Processor -- start 2024-03-28T18:00:13.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.997+0000 7f5b86c00640 1 -- start start 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b800723f0 0x7f5b8015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b80072ad0 0x7f5b8015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800b37a0 con 0x7f5b8015e030 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80162480 con 0x7f5b80072ad0 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b86c00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80071b90 con 0x7f5b800723f0 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b84c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b84c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42480/0 (socket says 172.21.15.62:42480) 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.998+0000 7f5b84c00640 1 -- 172.21.15.62:0/351527130 learned_addr learned my addr 172.21.15.62:0/351527130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b7fe00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b800723f0 0x7f5b8015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b7f400640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b80072ad0 0x7f5b8015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 -- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b800723f0 msgr2=0x7f5b8015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b800723f0 0x7f5b8015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 -- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b80072ad0 msgr2=0x7f5b8015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b80072ad0 0x7f5b8015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 -- 172.21.15.62:0/351527130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b801f3730 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b84c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c700 tx=0x7f5b7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:13.999+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78016020 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.000+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800aa50 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.000+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ce10 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.000+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8015a1a0 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.000+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b801602b0 con 0x7f5b8015e030 2024-03-28T18:00:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.001+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800680c0 con 0x7f5b8015e030 2024-03-28T18:00:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.002+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5b78019020 con 0x7f5b8015e030 2024-03-28T18:00:14.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.003+0000 7f5b77e00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 0x7f5b6006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.003+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b7808a9d0 con 0x7f5b8015e030 2024-03-28T18:00:14.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.004+0000 7f5b7fe00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 0x7f5b6006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.005+0000 7f5b7fe00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 0x7f5b6006e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b70036000 tx=0x7f5b7003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.008+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5b78020020 con 0x7f5b8015e030 2024-03-28T18:00:14.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.209+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b8006ba20 con 0x7f5b8015e030 2024-03-28T18:00:14.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.210+0000 7f5b77e00640 1 -- 172.21.15.62:0/351527130 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b78059800 con 0x7f5b8015e030 2024-03-28T18:00:14.210 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-28T18:00:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 msgr2=0x7f5b6006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 0x7f5b6006e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b70036000 tx=0x7f5b7003c040 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 msgr2=0x7f5b8015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c700 tx=0x7f5b7800cbc0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 shutdown_connections 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5b6006bc00 0x7f5b6006e0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8015e030 0x7f5b8015b8d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b80072ad0 0x7f5b8015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 --2- 172.21.15.62:0/351527130 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5b800723f0 0x7f5b8015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 >> 172.21.15.62:0/351527130 conn(0x7f5b800b1650 msgr2=0x7f5b800af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 shutdown_connections 2024-03-28T18:00:14.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.213+0000 7f5b86c00640 1 -- 172.21.15.62:0/351527130 wait complete. 2024-03-28T18:00:14.233 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T18:00:14.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:00:14.286 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:00:14.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.437+0000 7f1ba7200640 1 Processor -- start 2024-03-28T18:00:14.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.437+0000 7f1ba7200640 1 -- start start 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ba8072ab0 0x7f1ba815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ba806cd80 0x7f1ba806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80cebb0 con 0x7f1ba8072ab0 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80cee90 con 0x7f1ba806cd80 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba7200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80cf170 con 0x7f1ba81456d0 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ba8072ab0 0x7f1ba815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ba8072ab0 0x7f1ba815fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42486/0 (socket says 172.21.15.62:42486) 2024-03-28T18:00:14.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 -- 172.21.15.62:0/1211925182 learned_addr learned my addr 172.21.15.62:0/1211925182 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:14.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5e00640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 -- 172.21.15.62:0/1211925182 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 msgr2=0x7f1ba815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba6800640 1 --2- 172.21.15.62:0/1211925182 >> v2:172.21.15.81:3300/0 conn(0x7f1ba806cd80 0x7f1ba806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.439+0000 7f1ba5400640 1 -- 172.21.15.62:0/1211925182 >> v2:172.21.15.81:3300/0 conn(0x7f1ba806cd80 msgr2=0x7f1ba806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.440+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1211925182 >> v2:172.21.15.81:3300/0 conn(0x7f1ba806cd80 0x7f1ba806d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.440+0000 7f1ba5400640 1 -- 172.21.15.62:0/1211925182 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba80cf450 con 0x7f1ba8072ab0 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.440+0000 7f1ba5e00640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:14.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.441+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba8072ab0 0x7f1ba815fec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80b2ac0 tx=0x7f1b9402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe09f54a07a1003 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.441+0000 7f1ba4a00640 1 -- 172.21.15.62:0/1211925182 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94037070 con 0x7f1ba8072ab0 2024-03-28T18:00:14.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.441+0000 7f1ba4a00640 1 -- 172.21.15.62:0/1211925182 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94004230 con 0x7f1ba8072ab0 2024-03-28T18:00:14.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.441+0000 7f1ba4a00640 1 -- 172.21.15.62:0/1211925182 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94003610 con 0x7f1ba8072ab0 2024-03-28T18:00:14.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 -- 172.21.15.62:0/1211925182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba8072ab0 msgr2=0x7f1ba815fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba8072ab0 0x7f1ba815fec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80b2ac0 tx=0x7f1b9402b860 comp rx=0 tx=0).stop 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 -- 172.21.15.62:0/1211925182 shutdown_connections 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1211925182 >> v2:172.21.15.81:3300/0 conn(0x7f1ba806cd80 0x7f1ba806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba8072ab0 0x7f1ba815fec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1211925182 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 -- 172.21.15.62:0/1211925182 >> 172.21.15.62:0/1211925182 conn(0x7f1ba80b1730 msgr2=0x7f1ba80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 -- 172.21.15.62:0/1211925182 shutdown_connections 2024-03-28T18:00:14.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.442+0000 7f1ba7200640 1 -- 172.21.15.62:0/1211925182 wait complete. 2024-03-28T18:00:14.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.443+0000 7f1ba7200640 1 Processor -- start 2024-03-28T18:00:14.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.444+0000 7f1ba7200640 1 -- start start 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.444+0000 7f1ba7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.444+0000 7f1ba7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba8072ab0 0x7f1ba8139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba7200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba806d750 con 0x7f1ba806cd80 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80cf730 con 0x7f1ba8072ab0 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba7200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba806d8d0 con 0x7f1ba81456d0 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42490/0 (socket says 172.21.15.62:42490) 2024-03-28T18:00:14.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 -- 172.21.15.62:0/1714798115 learned_addr learned my addr 172.21.15.62:0/1714798115 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 -- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 msgr2=0x7f1ba815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba6800640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba8072ab0 0x7f1ba8139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 -- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba8072ab0 msgr2=0x7f1ba8139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba8072ab0 0x7f1ba8139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.445+0000 7f1ba5e00640 1 -- 172.21.15.62:0/1714798115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba813c480 con 0x7f1ba806cd80 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1ba5e00640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c7f0 tx=0x7f1b9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98015070 con 0x7f1ba806cd80 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b98007820 con 0x7f1ba806cd80 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98007bd0 con 0x7f1ba806cd80 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba813ca80 con 0x7f1ba806cd80 2024-03-28T18:00:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.446+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba813d040 con 0x7f1ba806cd80 2024-03-28T18:00:14.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.447+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba8073490 con 0x7f1ba806cd80 2024-03-28T18:00:14.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.448+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1b98018070 con 0x7f1ba806cd80 2024-03-28T18:00:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.449+0000 7f1b9e000640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 0x7f1b8806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.449+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b9808a940 con 0x7f1ba806cd80 2024-03-28T18:00:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.450+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 0x7f1b8806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.451+0000 7f1ba5400640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 0x7f1b8806e090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b94034000 tx=0x7f1b94008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.454+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f1b980596e0 con 0x7f1ba806cd80 2024-03-28T18:00:14.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.659+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1ba804f4e0 con 0x7f1ba806cd80 2024-03-28T18:00:14.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.660+0000 7f1b9e000640 1 -- 172.21.15.62:0/1714798115 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1b98058fe0 con 0x7f1ba806cd80 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:14.661 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 msgr2=0x7f1b8806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 0x7f1b8806e090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b94034000 tx=0x7f1b94008030 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 msgr2=0x7f1ba8138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c7f0 tx=0x7f1b9800ccb0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 shutdown_connections 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f1b8806bbd0 0x7f1b8806e090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f1ba81456d0 0x7f1ba815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ba8072ab0 0x7f1ba8139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 --2- 172.21.15.62:0/1714798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba806cd80 0x7f1ba8138b70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 >> 172.21.15.62:0/1714798115 conn(0x7f1ba80b1730 msgr2=0x7f1ba8071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 shutdown_connections 2024-03-28T18:00:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.663+0000 7f1ba7200640 1 -- 172.21.15.62:0/1714798115 wait complete. 2024-03-28T18:00:14.683 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T18:00:14.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T18:00:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.905+0000 7f2e0f200640 1 Processor -- start 2024-03-28T18:00:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.905+0000 7f2e0f200640 1 -- start start 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 0x7f2e1006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e100ae070 con 0x7f2e10066dd0 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0f200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e100ae310 con 0x7f2e101611f0 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100b59c0 con 0x7f2e100b00c0 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e10066dd0 0x7f2e100aead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60886/0 (socket says 172.21.15.62:60886) 2024-03-28T18:00:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.907+0000 7f2e0e800640 1 -- 172.21.15.62:0/520427657 learned_addr learned my addr 172.21.15.62:0/520427657 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0d400640 1 --2- 172.21.15.62:0/520427657 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 0x7f2e1006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764696861 0 0) 0x7f2e100ae070 con 0x7f2e10066dd0 2024-03-28T18:00:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df4003520 con 0x7f2e10066dd0 2024-03-28T18:00:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0d400640 1 -- 172.21.15.62:0/520427657 >> v1:172.21.15.81:6789/0 conn(0x7f2e101611f0 legacy=0x7f2e10162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025857981 0 0) 0x7f2df4003520 con 0x7f2e10066dd0 2024-03-28T18:00:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0d400640 1 -- 172.21.15.62:0/520427657 >> v1:172.21.15.62:6789/0 conn(0x7f2e10066dd0 legacy=0x7f2e100aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.908+0000 7f2e0d400640 1 -- 172.21.15.62:0/520427657 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100b5400 con 0x7f2e100b00c0 2024-03-28T18:00:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.910+0000 7f2e0d400640 1 --2- 172.21.15.62:0/520427657 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 0x7f2e1006d9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0097d0 tx=0x7f2dfc02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf3e3ad4c051c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.910+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc037070 con 0x7f2e100b00c0 2024-03-28T18:00:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc030510 con 0x7f2e100b00c0 2024-03-28T18:00:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0ca00640 1 -- 172.21.15.62:0/520427657 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e080019f0 con 0x7f2e100b00c0 2024-03-28T18:00:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 -- 172.21.15.62:0/520427657 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 msgr2=0x7f2e1006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 --2- 172.21.15.62:0/520427657 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 0x7f2e1006d9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0097d0 tx=0x7f2dfc02b770 comp rx=0 tx=0).stop 2024-03-28T18:00:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 -- 172.21.15.62:0/520427657 shutdown_connections 2024-03-28T18:00:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 --2- 172.21.15.62:0/520427657 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e100b00c0 0x7f2e1006d9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 -- 172.21.15.62:0/520427657 >> 172.21.15.62:0/520427657 conn(0x7f2e1006b6d0 msgr2=0x7f2e100b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 -- 172.21.15.62:0/520427657 shutdown_connections 2024-03-28T18:00:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.911+0000 7f2e0f200640 1 -- 172.21.15.62:0/520427657 wait complete. 2024-03-28T18:00:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.913+0000 7f2e0f200640 1 Processor -- start 2024-03-28T18:00:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.913+0000 7f2e0f200640 1 -- start start 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 0x7f2e1015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e101611f0 0x7f2e10159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006df30 con 0x7f2e10066dd0 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100ae5b0 con 0x7f2e100b00c0 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0f200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101615d0 con 0x7f2e101611f0 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42500/0 (socket says 172.21.15.62:42500) 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.914+0000 7f2e0de00640 1 -- 172.21.15.62:0/1484095717 learned_addr learned my addr 172.21.15.62:0/1484095717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0e800640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e101611f0 0x7f2e10159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0d400640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 0x7f2e1015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 -- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e101611f0 msgr2=0x7f2e10159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e101611f0 0x7f2e10159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 -- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 msgr2=0x7f2e1015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 0x7f2e1015d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 -- 172.21.15.62:0/1484095717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e101507e0 con 0x7f2e10066dd0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0d400640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 0x7f2e1015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0de00640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c500 tx=0x7f2e0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000052d0 con 0x7f2e10066dd0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00005470 con 0x7f2e10066dd0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.915+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e10150220 con 0x7f2e10066dd0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.916+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00012850 con 0x7f2e10066dd0 2024-03-28T18:00:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.916+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e10157d80 con 0x7f2e10066dd0 2024-03-28T18:00:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.917+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1004f400 con 0x7f2e10066dd0 2024-03-28T18:00:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.918+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2e000129f0 con 0x7f2e10066dd0 2024-03-28T18:00:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.919+0000 7f2e06000640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 0x7f2df006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.919+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e0008a380 con 0x7f2e10066dd0 2024-03-28T18:00:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.919+0000 7f2e0d400640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 0x7f2df006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.920+0000 7f2e0d400640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 0x7f2df006e090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc034000 tx=0x7f2dfc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:14.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:14.923+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2e000591d0 con 0x7f2e10066dd0 2024-03-28T18:00:15.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.129+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2e100b4ba0 con 0x7f2e10066dd0 2024-03-28T18:00:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.531+0000 7f2e06000640 1 -- 172.21.15.62:0/1484095717 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2e00058ad0 con 0x7f2e10066dd0 2024-03-28T18:00:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 msgr2=0x7f2df006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 0x7f2df006e090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc034000 tx=0x7f2dfc008030 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 msgr2=0x7f2e1014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c500 tx=0x7f2e0000c9c0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 shutdown_connections 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f2df006bbd0 0x7f2df006e090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f2e101611f0 0x7f2e10159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e100b00c0 0x7f2e1015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 --2- 172.21.15.62:0/1484095717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e10066dd0 0x7f2e1014d6b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.535+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 >> 172.21.15.62:0/1484095717 conn(0x7f2e1006b6d0 msgr2=0x7f2e1006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.536+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 shutdown_connections 2024-03-28T18:00:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.536+0000 7f2e0f200640 1 -- 172.21.15.62:0/1484095717 wait complete. 2024-03-28T18:00:15.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T18:00:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.766+0000 7fbc6a000640 1 Processor -- start 2024-03-28T18:00:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.766+0000 7fbc6a000640 1 -- start start 2024-03-28T18:00:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.768+0000 7fbc6a000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc640d1ca0 0x7fbc640d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.768+0000 7fbc6a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc640b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.768+0000 7fbc6a000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640d2330 con 0x7fbc640d3b40 2024-03-28T18:00:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.768+0000 7fbc6a000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640d3910 con 0x7fbc640d1ca0 2024-03-28T18:00:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.768+0000 7fbc6a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640d20e0 con 0x7fbc640d3f20 2024-03-28T18:00:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.769+0000 7fbc63e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc640d1ca0 0x7fbc640d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.769+0000 7fbc69600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc640b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.769+0000 7fbc63e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc640d1ca0 0x7fbc640d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36476/0 (socket says 172.21.15.62:36476) 2024-03-28T18:00:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.769+0000 7fbc63e00640 1 -- 172.21.15.62:0/2184953417 learned_addr learned my addr 172.21.15.62:0/2184953417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.769+0000 7fbc68c00640 1 --1- 172.21.15.62:0/2184953417 >> v1:172.21.15.81:6789/0 conn(0x7fbc640d3b40 0x7fbc640b9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.62:52906/0 (socket says 172.21.15.62:52906) 2024-03-28T18:00:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.770+0000 7fbc63e00640 1 -- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 msgr2=0x7fbc640b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.770+0000 7fbc63e00640 1 --2- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc640b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.770+0000 7fbc63e00640 1 -- 172.21.15.62:0/2184953417 >> v1:172.21.15.81:6789/0 conn(0x7fbc640d3b40 legacy=0x7fbc640b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.770+0000 7fbc63e00640 1 -- 172.21.15.62:0/2184953417 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640d2610 con 0x7fbc640d1ca0 2024-03-28T18:00:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.771+0000 7fbc63e00640 1 --2- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d1ca0 0x7fbc640d33d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009870 tx=0x7fbc5002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e5dc4a3e2f39f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.771+0000 7fbc63400640 1 -- 172.21.15.62:0/2184953417 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50034070 con 0x7fbc640d1ca0 2024-03-28T18:00:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.771+0000 7fbc63400640 1 -- 172.21.15.62:0/2184953417 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc50030840 con 0x7fbc640d1ca0 2024-03-28T18:00:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.771+0000 7fbc63400640 1 -- 172.21.15.62:0/2184953417 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50030c10 con 0x7fbc640d1ca0 2024-03-28T18:00:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 -- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d1ca0 msgr2=0x7fbc640d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d1ca0 0x7fbc640d33d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009870 tx=0x7fbc5002b980 comp rx=0 tx=0).stop 2024-03-28T18:00:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 -- 172.21.15.62:0/2184953417 shutdown_connections 2024-03-28T18:00:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc640b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2184953417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d1ca0 0x7fbc640d33d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 -- 172.21.15.62:0/2184953417 >> 172.21.15.62:0/2184953417 conn(0x7fbc64071ae0 msgr2=0x7fbc64071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 -- 172.21.15.62:0/2184953417 shutdown_connections 2024-03-28T18:00:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.772+0000 7fbc6a000640 1 -- 172.21.15.62:0/2184953417 wait complete. 2024-03-28T18:00:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.775+0000 7fbc6a000640 1 Processor -- start 2024-03-28T18:00:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.775+0000 7fbc6a000640 1 -- start start 2024-03-28T18:00:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d3b40 0x7fbc64154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc641f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640d2eb0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640d2bd0 con 0x7fbc640d3b40 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc6a000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415e0f0 con 0x7fbc640d3f20 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.776+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42508/0 (socket says 172.21.15.62:42508) 2024-03-28T18:00:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 -- 172.21.15.62:0/2838469266 learned_addr learned my addr 172.21.15.62:0/2838469266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc69600640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc641f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc63e00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d3b40 0x7fbc64154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 -- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 msgr2=0x7fbc641f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc641f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 -- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d3b40 msgr2=0x7fbc64154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d3b40 0x7fbc64154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 -- 172.21.15.62:0/2838469266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640b5540 con 0x7fbc640d1ca0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc69600640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc641f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.777+0000 7fbc68c00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c430 tx=0x7fbc5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.778+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580074e0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.778+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc58007680 con 0x7fbc640d1ca0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.778+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58004a00 con 0x7fbc640d1ca0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.778+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc641f7db0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.778+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc640b57e0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.779+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc640cfbb0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.780+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbc5801a020 con 0x7fbc640d1ca0 2024-03-28T18:00:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.782+0000 7fbc60c00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 0x7fbc4406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.782+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbc5808a8e0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.783+0000 7fbc63e00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 0x7fbc4406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.784+0000 7fbc63e00640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 0x7fbc4406e090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc50036000 tx=0x7fbc5003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.786+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fbc640cfbb0 con 0x7fbc640d1ca0 2024-03-28T18:00:15.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:15.989+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc640d43b0 con 0x7fbc640d1ca0 2024-03-28T18:00:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.547+0000 7fbc60c00640 1 -- 172.21.15.62:0/2838469266 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbc640d43b0 con 0x7fbc640d1ca0 2024-03-28T18:00:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-03-28T18:00:16.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 msgr2=0x7fbc4406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 0x7fbc4406e090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc50036000 tx=0x7fbc5003b040 comp rx=0 tx=0).stop 2024-03-28T18:00:16.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 msgr2=0x7fbc640b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c430 tx=0x7fbc5800c8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 shutdown_connections 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fbc4406bbd0 0x7fbc4406e090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbc640d3f20 0x7fbc641f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc640d3b40 0x7fbc64154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 --2- 172.21.15.62:0/2838469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc640d1ca0 0x7fbc640b5fe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 >> 172.21.15.62:0/2838469266 conn(0x7fbc64071ae0 msgr2=0x7fbc6415ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 shutdown_connections 2024-03-28T18:00:16.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.551+0000 7fbc6a000640 1 -- 172.21.15.62:0/2838469266 wait complete. 2024-03-28T18:00:16.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T18:00:16.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.773+0000 7fba67200640 1 Processor -- start 2024-03-28T18:00:16.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.773+0000 7fba67200640 1 -- start start 2024-03-28T18:00:16.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba67200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba600aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba67200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6015d1b0 con 0x7fba60072ad0 2024-03-28T18:00:16.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba67200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba60160660 con 0x7fba60072690 2024-03-28T18:00:16.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba67200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba600af040 con 0x7fba600ae720 2024-03-28T18:00:16.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba65200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fba60072ad0 0x7fba6015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60896/0 (socket says 172.21.15.62:60896) 2024-03-28T18:00:16.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.774+0000 7fba65200640 1 -- 172.21.15.62:0/3028882985 learned_addr learned my addr 172.21.15.62:0/3028882985 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5fe00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba600aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763255999 0 0) 0x7fba6015d1b0 con 0x7fba60072ad0 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba44003520 con 0x7fba60072ad0 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282258415 0 0) 0x7fba44003520 con 0x7fba60072ad0 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 msgr2=0x7fba600aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:16.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba600aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 >> v1:172.21.15.81:6789/0 conn(0x7fba60072690 legacy=0x7fba6015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba60070d50 con 0x7fba60072ad0 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7fba580019f0 con 0x7fba60072ad0 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 >> v1:172.21.15.62:6789/0 conn(0x7fba60072ad0 legacy=0x7fba6015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.775+0000 7fba5ea00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba440039e0 0x7fba44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5ea00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba44004800 0x7fba44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5ea00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba600af040 con 0x7fba44004800 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4400ba70 con 0x7fba440039e0 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4400bc70 con 0x7fba600ae720 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba65200640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5fe00640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba440039e0 0x7fba44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.776+0000 7fba5f400640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba44004800 0x7fba44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.777+0000 7fba5f400640 1 -- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 msgr2=0x7fba440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.777+0000 7fba5f400640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba600ae720 0x7fba440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.777+0000 7fba5f400640 1 -- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba440039e0 msgr2=0x7fba44003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.777+0000 7fba5f400640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba440039e0 0x7fba44003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.777+0000 7fba5f400640 1 -- 172.21.15.62:0/3028882985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44003520 con 0x7fba44004800 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.778+0000 7fba5f400640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba44004800 0x7fba44004c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001720 tx=0x7fba44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ff08d7df40201c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.778+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c001370 con 0x7fba44004800 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.778+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c0052d0 con 0x7fba44004800 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.778+0000 7fba5ea00640 1 -- 172.21.15.62:0/3028882985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c003830 con 0x7fba44004800 2024-03-28T18:00:16.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 -- 172.21.15.62:0/3028882985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba44004800 msgr2=0x7fba44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 --2- 172.21.15.62:0/3028882985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba44004800 0x7fba44004c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001720 tx=0x7fba44005830 comp rx=0 tx=0).stop 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba65200640 1 -- 172.21.15.62:0/3028882985 reap_dead start 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 -- 172.21.15.62:0/3028882985 shutdown_connections 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 -- 172.21.15.62:0/3028882985 >> 172.21.15.62:0/3028882985 conn(0x7fba600b1650 msgr2=0x7fba600b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 -- 172.21.15.62:0/3028882985 shutdown_connections 2024-03-28T18:00:16.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.779+0000 7fba67200640 1 -- 172.21.15.62:0/3028882985 wait complete. 2024-03-28T18:00:16.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.781+0000 7fba67200640 1 Processor -- start 2024-03-28T18:00:16.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.781+0000 7fba67200640 1 -- start start 2024-03-28T18:00:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba44004800 0x7fba601434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba601e57b0 0x7fba601e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba600715f0 con 0x7fba60151970 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60071310 con 0x7fba44004800 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba67200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba600718d0 con 0x7fba601e57b0 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42524/0 (socket says 172.21.15.62:42524) 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.782+0000 7fba5f400640 1 -- 172.21.15.62:0/3232312988 learned_addr learned my addr 172.21.15.62:0/3232312988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5fe00640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba44004800 0x7fba601434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba65200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba601e57b0 0x7fba601e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 -- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba601e57b0 msgr2=0x7fba601e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba601e57b0 0x7fba601e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 -- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba44004800 msgr2=0x7fba601434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba44004800 0x7fba601434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 -- 172.21.15.62:0/3232312988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba601e74d0 con 0x7fba60151970 2024-03-28T18:00:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba5f400640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba4c008fd0 tx=0x7fba4c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c041070 con 0x7fba60151970 2024-03-28T18:00:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c005480 con 0x7fba60151970 2024-03-28T18:00:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c007a30 con 0x7fba60151970 2024-03-28T18:00:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.783+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601e6180 con 0x7fba60151970 2024-03-28T18:00:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.784+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601e6710 con 0x7fba60151970 2024-03-28T18:00:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.785+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba600680c0 con 0x7fba60151970 2024-03-28T18:00:16.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.786+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fba4c044070 con 0x7fba60151970 2024-03-28T18:00:16.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.787+0000 7fba57e00640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 0x7fba4006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:16.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.787+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fba4c0ac640 con 0x7fba60151970 2024-03-28T18:00:16.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.788+0000 7fba5fe00640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 0x7fba4006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:16.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.789+0000 7fba5fe00640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 0x7fba4006e090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba50009a80 tx=0x7fba50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:16.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:16.793+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fba4c07b180 con 0x7fba60151970 2024-03-28T18:00:17.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.035+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fba600b2420 con 0x7fba60151970 2024-03-28T18:00:17.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.552+0000 7fba57e00640 1 -- 172.21.15.62:0/3232312988 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fba4c07aa80 con 0x7fba60151970 2024-03-28T18:00:17.552 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 4 and data pool 5 2024-03-28T18:00:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 msgr2=0x7fba4006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 0x7fba4006e090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba50009a80 tx=0x7fba50008040 comp rx=0 tx=0).stop 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 msgr2=0x7fba60151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba4c008fd0 tx=0x7fba4c001120 comp rx=0 tx=0).stop 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 shutdown_connections 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fba4006bbd0 0x7fba4006e090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fba601e57b0 0x7fba601e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba60151970 0x7fba60151df0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba4c008fd0 tx=0x7fba4c001120 comp rx=0 tx=0).stop 2024-03-28T18:00:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 --2- 172.21.15.62:0/3232312988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba44004800 0x7fba601434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.556+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 >> 172.21.15.62:0/3232312988 conn(0x7fba600b1650 msgr2=0x7fba6006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.557+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 shutdown_connections 2024-03-28T18:00:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.557+0000 7fba67200640 1 -- 172.21.15.62:0/3232312988 wait complete. 2024-03-28T18:00:17.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.784+0000 7f56f1200640 1 Processor -- start 2024-03-28T18:00:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.784+0000 7f56f1200640 1 -- start start 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.785+0000 7f56f1200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.785+0000 7f56f1200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56ec071d90 0x7f56ec072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.785+0000 7f56f1200640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0b30d0 con 0x7f56ec0ae370 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.785+0000 7f56f1200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0b33b0 con 0x7f56ec071d90 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.785+0000 7f56f1200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0b3690 con 0x7f56ec0719b0 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56ec071d90 0x7f56ec072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56ec071d90 0x7f56ec072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42536/0 (socket says 172.21.15.62:42536) 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 -- 172.21.15.62:0/435072559 learned_addr learned my addr 172.21.15.62:0/435072559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56e9e00640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 -- 172.21.15.62:0/435072559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 msgr2=0x7f56ec0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 -- 172.21.15.62:0/435072559 >> v1:172.21.15.81:6789/0 conn(0x7f56ec0ae370 legacy=0x7f56ec0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.786+0000 7f56ea800640 1 -- 172.21.15.62:0/435072559 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0b3970 con 0x7f56ec071d90 2024-03-28T18:00:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.787+0000 7f56ea800640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec071d90 0x7f56ec072220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ed0db4508a0c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56e9400640 1 -- 172.21.15.62:0/435072559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e402ca70 con 0x7f56ec071d90 2024-03-28T18:00:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56e9400640 1 -- 172.21.15.62:0/435072559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e402cc10 con 0x7f56ec071d90 2024-03-28T18:00:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56e9400640 1 -- 172.21.15.62:0/435072559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40343f0 con 0x7f56ec071d90 2024-03-28T18:00:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 -- 172.21.15.62:0/435072559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec071d90 msgr2=0x7f56ec072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec071d90 0x7f56ec072220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e402bed0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 -- 172.21.15.62:0/435072559 shutdown_connections 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec071d90 0x7f56ec072220 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 --2- 172.21.15.62:0/435072559 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 -- 172.21.15.62:0/435072559 >> 172.21.15.62:0/435072559 conn(0x7f56ec066fb0 msgr2=0x7f56ec0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.788+0000 7f56f1200640 1 -- 172.21.15.62:0/435072559 shutdown_connections 2024-03-28T18:00:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.789+0000 7f56f1200640 1 -- 172.21.15.62:0/435072559 wait complete. 2024-03-28T18:00:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.790+0000 7f56f1200640 1 Processor -- start 2024-03-28T18:00:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.790+0000 7f56f1200640 1 -- start start 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec09bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 0x7f56ec144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0b46b0 con 0x7f56ec0ae370 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec072760 con 0x7f56ec071d90 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.791+0000 7f56f1200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec06ccb0 con 0x7f56ec0719b0 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42542/0 (socket says 172.21.15.62:42542) 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 -- 172.21.15.62:0/641117103 learned_addr learned my addr 172.21.15.62:0/641117103 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56e9e00640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec09bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56e3e00640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 0x7f56ec144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 -- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 msgr2=0x7f56ec09bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec09bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 -- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 msgr2=0x7f56ec144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 0x7f56ec144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 -- 172.21.15.62:0/641117103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec09c300 con 0x7f56ec0ae370 2024-03-28T18:00:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56e3e00640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 0x7f56ec144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.792+0000 7f56ea800640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f56e4036040 tx=0x7f56e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.793+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4043020 con 0x7f56ec0ae370 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.793+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e403e450 con 0x7f56ec0ae370 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.793+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4009510 con 0x7f56ec0ae370 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.793+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec09c5e0 con 0x7f56ec0ae370 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.793+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec09cba0 con 0x7f56ec0ae370 2024-03-28T18:00:17.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.794+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec0680c0 con 0x7f56ec0ae370 2024-03-28T18:00:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.795+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f56e4046020 con 0x7f56ec0ae370 2024-03-28T18:00:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.796+0000 7f56e2000640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 0x7f56c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.796+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56e40ad270 con 0x7f56ec0ae370 2024-03-28T18:00:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.796+0000 7f56e9e00640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 0x7f56c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.798+0000 7f56e9e00640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 0x7f56c806e090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.801+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f56e404d030 con 0x7f56ec0ae370 2024-03-28T18:00:17.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:17.999+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56ec067390 con 0x7f56ec0ae370 2024-03-28T18:00:18.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.002+0000 7f56e2000640 1 -- 172.21.15.62:0/641117103 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16634 (secure 0 0 0) 0x7f56e407bd50 con 0x7f56ec0ae370 2024-03-28T18:00:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:18.003 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T18:00:17.036883+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:00:15.184773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T18:00:16.044475+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 msgr2=0x7f56c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 0x7f56c806e090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc008040 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 msgr2=0x7f56ec155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f56e4036040 tx=0x7f56e4004a40 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 shutdown_connections 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f56c806bbd0 0x7f56c806e090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56ec0ae370 0x7f56ec155200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56ec071d90 0x7f56ec144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 --2- 172.21.15.62:0/641117103 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f56ec0719b0 0x7f56ec09bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.005+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 >> 172.21.15.62:0/641117103 conn(0x7f56ec066fb0 msgr2=0x7f56ec0ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.006+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 shutdown_connections 2024-03-28T18:00:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.006+0000 7f56f1200640 1 -- 172.21.15.62:0/641117103 wait complete. 2024-03-28T18:00:18.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:18.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.237+0000 7f5c77e00640 1 Processor -- start 2024-03-28T18:00:18.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.238+0000 7f5c77e00640 1 -- start start 2024-03-28T18:00:18.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.239+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.239+0000 7f5c77e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c780b5e00 0x7f5c780b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.239+0000 7f5c77e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780b6e60 con 0x7f5c780b99f0 2024-03-28T18:00:18.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.239+0000 7f5c77e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780b6c30 con 0x7f5c780b5e00 2024-03-28T18:00:18.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.239+0000 7f5c77e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b7100 con 0x7f5c7815dd60 2024-03-28T18:00:18.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.240+0000 7f5c76a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.240+0000 7f5c77400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c780b5e00 0x7f5c780b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.240+0000 7f5c76a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:56816/0 (socket says 172.21.15.62:56816) 2024-03-28T18:00:18.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.240+0000 7f5c76a00640 1 -- 172.21.15.62:0/3200521180 learned_addr learned my addr 172.21.15.62:0/3200521180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:18.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.241+0000 7f5c76a00640 1 -- 172.21.15.62:0/3200521180 >> v2:172.21.15.81:3300/0 conn(0x7f5c780b5e00 msgr2=0x7f5c780b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:18.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.241+0000 7f5c76a00640 1 --2- 172.21.15.62:0/3200521180 >> v2:172.21.15.81:3300/0 conn(0x7f5c780b5e00 0x7f5c780b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.241+0000 7f5c76a00640 1 -- 172.21.15.62:0/3200521180 >> v1:172.21.15.81:6789/0 conn(0x7f5c780b99f0 legacy=0x7f5c780b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.241+0000 7f5c76a00640 1 -- 172.21.15.62:0/3200521180 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780b73e0 con 0x7f5c7815dd60 2024-03-28T18:00:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.242+0000 7f5c76a00640 1 --2- 172.21.15.62:0/3200521180 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009870 tx=0x7f5c6802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ebebfa27f34f441 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.242+0000 7f5c75600640 1 -- 172.21.15.62:0/3200521180 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6802f830 con 0x7f5c7815dd60 2024-03-28T18:00:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.242+0000 7f5c75600640 1 -- 172.21.15.62:0/3200521180 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6802f9d0 con 0x7f5c7815dd60 2024-03-28T18:00:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.242+0000 7f5c75600640 1 -- 172.21.15.62:0/3200521180 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6802fda0 con 0x7f5c7815dd60 2024-03-28T18:00:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 -- 172.21.15.62:0/3200521180 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 msgr2=0x7f5c780b94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 --2- 172.21.15.62:0/3200521180 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009870 tx=0x7f5c6802ba10 comp rx=0 tx=0).stop 2024-03-28T18:00:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 -- 172.21.15.62:0/3200521180 shutdown_connections 2024-03-28T18:00:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 --2- 172.21.15.62:0/3200521180 >> v2:172.21.15.81:3300/0 conn(0x7f5c780b5e00 0x7f5c780b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 --2- 172.21.15.62:0/3200521180 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c780b94b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 -- 172.21.15.62:0/3200521180 >> 172.21.15.62:0/3200521180 conn(0x7f5c78071ae0 msgr2=0x7f5c78071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 -- 172.21.15.62:0/3200521180 shutdown_connections 2024-03-28T18:00:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.243+0000 7f5c77e00640 1 -- 172.21.15.62:0/3200521180 wait complete. 2024-03-28T18:00:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.246+0000 7f5c77e00640 1 Processor -- start 2024-03-28T18:00:18.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.246+0000 7f5c77e00640 1 -- start start 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780b99f0 0x7f5c780cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c78152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815e140 con 0x7f5c780b5e00 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b79a0 con 0x7f5c780b99f0 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c77e00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b6370 con 0x7f5c7815dd60 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c76a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c76a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42550/0 (socket says 172.21.15.62:42550) 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.247+0000 7f5c76a00640 1 -- 172.21.15.62:0/2446224329 learned_addr learned my addr 172.21.15.62:0/2446224329 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c77400640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c78152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76000640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780b99f0 0x7f5c780cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 -- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 msgr2=0x7f5c78152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c78152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 -- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780b99f0 msgr2=0x7f5c780cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780b99f0 0x7f5c780cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 -- 172.21.15.62:0/2446224329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780cb720 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.248+0000 7f5c76a00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c6802ba50 tx=0x7f5c68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.249+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68047070 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.249+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68037c40 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.249+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68042480 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.249+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780cba00 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.249+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780cbfc0 con 0x7f5c780b5e00 2024-03-28T18:00:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.250+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c780d00e0 con 0x7f5c780b5e00 2024-03-28T18:00:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.251+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5c6804a070 con 0x7f5c780b5e00 2024-03-28T18:00:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.252+0000 7f5c6ea00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 0x7f5c5406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.252+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c680acbf0 con 0x7f5c780b5e00 2024-03-28T18:00:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.253+0000 7f5c76000640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 0x7f5c5406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.254+0000 7f5c76000640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 0x7f5c5406e090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007fd0 tx=0x7f5c60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.257+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f5c780d00e0 con 0x7f5c780b5e00 2024-03-28T18:00:18.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.461+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c780b2de0 con 0x7f5c780b5e00 2024-03-28T18:00:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.463+0000 7f5c6ea00640 1 -- 172.21.15.62:0/2446224329 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16634 (secure 0 0 0) 0x7f5c68033030 con 0x7f5c780b5e00 2024-03-28T18:00:18.463 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:18.464 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T18:00:17.036883+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:00:15.184773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T18:00:16.044475+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:18.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 msgr2=0x7f5c5406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 0x7f5c5406e090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007fd0 tx=0x7f5c60009040 comp rx=0 tx=0).stop 2024-03-28T18:00:18.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 msgr2=0x7f5c78150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c6802ba50 tx=0x7f5c68004730 comp rx=0 tx=0).stop 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 shutdown_connections 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f5c5406bbd0 0x7f5c5406e090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f5c7815dd60 0x7f5c78152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780b99f0 0x7f5c780cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 --2- 172.21.15.62:0/2446224329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c780b5e00 0x7f5c78150240 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 >> 172.21.15.62:0/2446224329 conn(0x7f5c78071ae0 msgr2=0x7f5c780b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 shutdown_connections 2024-03-28T18:00:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.466+0000 7f5c77e00640 1 -- 172.21.15.62:0/2446224329 wait complete. 2024-03-28T18:00:18.485 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T18:00:18.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.709+0000 7f0069400640 1 Processor -- start 2024-03-28T18:00:18.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.710+0000 7f0069400640 1 -- start start 2024-03-28T18:00:18.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.711+0000 7f0069400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f00640d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.711+0000 7f0069400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f00640d0d50 0x7f00640b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.711+0000 7f0069400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00640d2510 con 0x7f00640ce780 2024-03-28T18:00:18.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.711+0000 7f0069400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640d27b0 con 0x7f00640d0d50 2024-03-28T18:00:18.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.711+0000 7f0069400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00640d2a90 con 0x7f006415e210 2024-03-28T18:00:18.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0063200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f00640d0d50 0x7f00640b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f005be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f00640ce780 0x7f00640d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60900/0 (socket says 172.21.15.62:60900) 2024-03-28T18:00:18.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f005be00640 1 -- 172.21.15.62:0/2376187374 learned_addr learned my addr 172.21.15.62:0/2376187374 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0062800640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f00640d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0061e00640 1 -- 172.21.15.62:0/2376187374 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017620975 0 0) 0x7f00640d2510 con 0x7f00640ce780 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0061e00640 1 -- 172.21.15.62:0/2376187374 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0044003520 con 0x7f00640ce780 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0063200640 1 -- 172.21.15.62:0/2376187374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 msgr2=0x7f00640d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0063200640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f00640d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0063200640 1 -- 172.21.15.62:0/2376187374 >> v1:172.21.15.62:6789/0 conn(0x7f00640ce780 legacy=0x7f00640d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.712+0000 7f0063200640 1 -- 172.21.15.62:0/2376187374 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640d2d70 con 0x7f00640d0d50 2024-03-28T18:00:18.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.714+0000 7f0063200640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f00640b70e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f005c009d90 tx=0x7f005c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6905ad42c34ed0b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.714+0000 7f0061e00640 1 -- 172.21.15.62:0/2376187374 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c039070 con 0x7f00640d0d50 2024-03-28T18:00:18.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.714+0000 7f0061e00640 1 -- 172.21.15.62:0/2376187374 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c02cc50 con 0x7f00640d0d50 2024-03-28T18:00:18.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.714+0000 7f0061e00640 1 -- 172.21.15.62:0/2376187374 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c034440 con 0x7f00640d0d50 2024-03-28T18:00:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 -- 172.21.15.62:0/2376187374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 msgr2=0x7f00640b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f00640b70e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f005c009d90 tx=0x7f005c02bea0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 -- 172.21.15.62:0/2376187374 shutdown_connections 2024-03-28T18:00:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f00640b70e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 --2- 172.21.15.62:0/2376187374 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f00640d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 -- 172.21.15.62:0/2376187374 >> 172.21.15.62:0/2376187374 conn(0x7f0064066dd0 msgr2=0x7f00640671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:18.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 -- 172.21.15.62:0/2376187374 shutdown_connections 2024-03-28T18:00:18.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.715+0000 7f0069400640 1 -- 172.21.15.62:0/2376187374 wait complete. 2024-03-28T18:00:18.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.719+0000 7f0069400640 1 Processor -- start 2024-03-28T18:00:18.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.719+0000 7f0069400640 1 -- start start 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00640ce780 0x7f00641f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f006415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006415e5f0 con 0x7f00640d0d50 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640d3330 con 0x7f00640ce780 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0069400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00640b7620 con 0x7f006415e210 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f005be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f005be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42572/0 (socket says 172.21.15.62:42572) 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f005be00640 1 -- 172.21.15.62:0/654163530 learned_addr learned my addr 172.21.15.62:0/654163530 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f0063200640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f006415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f005be00640 1 -- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 msgr2=0x7f006415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.722+0000 7f005be00640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f006415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005be00640 1 -- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00640ce780 msgr2=0x7f00641f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f0062800640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00640ce780 0x7f00641f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005be00640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00640ce780 0x7f00641f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005be00640 1 -- 172.21.15.62:0/654163530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00641f45e0 con 0x7f00640d0d50 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005be00640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c430 tx=0x7f004c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c017070 con 0x7f00640d0d50 2024-03-28T18:00:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c005ce0 con 0x7f00640d0d50 2024-03-28T18:00:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0072f0 con 0x7f00640d0d50 2024-03-28T18:00:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.723+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00641f6da0 con 0x7f00640d0d50 2024-03-28T18:00:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.724+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00641f7360 con 0x7f00640d0d50 2024-03-28T18:00:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.725+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00640680c0 con 0x7f00640d0d50 2024-03-28T18:00:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.726+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f004c01a070 con 0x7f00640d0d50 2024-03-28T18:00:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.727+0000 7f005aa00640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 0x7f004006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.727+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f004c089f20 con 0x7f00640d0d50 2024-03-28T18:00:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.727+0000 7f0062800640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 0x7f004006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.729+0000 7f0062800640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 0x7f004006e090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0054009970 tx=0x7f0054008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:18.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.732+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f004c058a00 con 0x7f00640d0d50 2024-03-28T18:00:18.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:18.979+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f00640b9990 con 0x7f00640d0d50 2024-03-28T18:00:19.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.597+0000 7f005aa00640 1 -- 172.21.15.62:0/654163530 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f004c058300 con 0x7f00640d0d50 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.601+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 msgr2=0x7f004006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.601+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 0x7f004006e090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0054009970 tx=0x7f0054008040 comp rx=0 tx=0).stop 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.601+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 msgr2=0x7f006415e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.601+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c430 tx=0x7f004c00c8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 shutdown_connections 2024-03-28T18:00:19.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f004006bbd0 0x7f004006e090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f006415e210 0x7f006415ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00640d0d50 0x7f006415e800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 --2- 172.21.15.62:0/654163530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00640ce780 0x7f00641f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 >> 172.21.15.62:0/654163530 conn(0x7f0064066dd0 msgr2=0x7f00640d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 shutdown_connections 2024-03-28T18:00:19.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.602+0000 7f0069400640 1 -- 172.21.15.62:0/654163530 wait complete. 2024-03-28T18:00:19.624 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T18:00:19.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T18:00:19.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.890+0000 7fe4bda00640 1 Processor -- start 2024-03-28T18:00:19.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.890+0000 7fe4bda00640 1 -- start start 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bda00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4b80cec50 0x7fe4b80ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 0x7fe4b80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bda00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b80cf570 con 0x7fe4b80b3210 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bda00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b8161bb0 con 0x7fe4b80cec50 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bda00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8161e90 con 0x7fe4b815e2c0 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4b80cec50 0x7fe4b80ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bd000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe4b80b3210 0x7fe4b80b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60910/0 (socket says 172.21.15.62:60910) 2024-03-28T18:00:19.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.892+0000 7fe4bd000640 1 -- 172.21.15.62:0/1201993527 learned_addr learned my addr 172.21.15.62:0/1201993527 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7400640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 0x7fe4b80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7e00640 1 -- 172.21.15.62:0/1201993527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 msgr2=0x7fe4b80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7e00640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 0x7fe4b80cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7e00640 1 -- 172.21.15.62:0/1201993527 >> v1:172.21.15.62:6789/0 conn(0x7fe4b80b3210 legacy=0x7fe4b80b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7e00640 1 -- 172.21.15.62:0/1201993527 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8162170 con 0x7fe4b80cec50 2024-03-28T18:00:19.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.893+0000 7fe4b7400640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 0x7fe4b80cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.894+0000 7fe4b7e00640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b80cec50 0x7fe4b80ce570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009d90 tx=0x7fe4a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18047b693c730349 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.894+0000 7fe4b6a00640 1 -- 172.21.15.62:0/1201993527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8039070 con 0x7fe4b80cec50 2024-03-28T18:00:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.894+0000 7fe4b6a00640 1 -- 172.21.15.62:0/1201993527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a802cc60 con 0x7fe4b80cec50 2024-03-28T18:00:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.894+0000 7fe4b6a00640 1 -- 172.21.15.62:0/1201993527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8034660 con 0x7fe4b80cec50 2024-03-28T18:00:19.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 -- 172.21.15.62:0/1201993527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b80cec50 msgr2=0x7fe4b80ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b80cec50 0x7fe4b80ce570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009d90 tx=0x7fe4a802beb0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 -- 172.21.15.62:0/1201993527 shutdown_connections 2024-03-28T18:00:19.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b815e2c0 0x7fe4b80cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 --2- 172.21.15.62:0/1201993527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b80cec50 0x7fe4b80ce570 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 -- 172.21.15.62:0/1201993527 >> 172.21.15.62:0/1201993527 conn(0x7fe4b8066fb0 msgr2=0x7fe4b80673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:19.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 -- 172.21.15.62:0/1201993527 shutdown_connections 2024-03-28T18:00:19.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.895+0000 7fe4bda00640 1 -- 172.21.15.62:0/1201993527 wait complete. 2024-03-28T18:00:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.897+0000 7fe4bda00640 1 Processor -- start 2024-03-28T18:00:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.898+0000 7fe4bda00640 1 -- start start 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b80b3210 0x7fe4b80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4b80cec50 0x7fe4b814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8162a10 con 0x7fe4b814d130 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8162730 con 0x7fe4b80cec50 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bda00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80d2fc0 con 0x7fe4b80b3210 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.899+0000 7fe4bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42600/0 (socket says 172.21.15.62:42600) 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 -- 172.21.15.62:0/2101019421 learned_addr learned my addr 172.21.15.62:0/2101019421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4b7e00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b80b3210 0x7fe4b80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4b7400640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4b80cec50 0x7fe4b814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 -- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b80b3210 msgr2=0x7fe4b80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b80b3210 0x7fe4b80c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 -- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4b80cec50 msgr2=0x7fe4b814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4b80cec50 0x7fe4b814cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 -- 172.21.15.62:0/2101019421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80b4560 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.900+0000 7fe4bd000640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c390 tx=0x7fe4b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.901+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0017070 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.901+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0005ce0 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.901+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00072d0 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.901+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b80b4840 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.901+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b80b4e00 con 0x7fe4b814d130 2024-03-28T18:00:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.902+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b804f400 con 0x7fe4b814d130 2024-03-28T18:00:19.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.903+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe4b001a070 con 0x7fe4b814d130 2024-03-28T18:00:19.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.904+0000 7fe4afe00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 0x7fe49406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:19.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.904+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4b0089cf0 con 0x7fe4b814d130 2024-03-28T18:00:19.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.905+0000 7fe4b7e00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 0x7fe49406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:19.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.906+0000 7fe4b7e00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 0x7fe49406e0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8036000 tx=0x7fe4a8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:19.912+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe4b008e070 con 0x7fe4b814d130 2024-03-28T18:00:20.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.153+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe4b80d0fe0 con 0x7fe4b814d130 2024-03-28T18:00:20.504 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-03-28T18:00:20.504+0000 7fbb87a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T18:00:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.610+0000 7fe4afe00640 1 -- 172.21.15.62:0/2101019421 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4b0058110 con 0x7fe4b814d130 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.614+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 msgr2=0x7fe49406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.614+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 0x7fe49406e0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8036000 tx=0x7fe4a8030000 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.614+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 msgr2=0x7fe4b80b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.614+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c390 tx=0x7fe4b000c850 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 shutdown_connections 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe49406bc00 0x7fe49406e0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4b814d130 0x7fe4b80b3810 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4b80cec50 0x7fe4b814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 --2- 172.21.15.62:0/2101019421 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe4b80b3210 0x7fe4b80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 >> 172.21.15.62:0/2101019421 conn(0x7fe4b8066fb0 msgr2=0x7fe4b80d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 shutdown_connections 2024-03-28T18:00:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.615+0000 7fe4bda00640 1 -- 172.21.15.62:0/2101019421 wait complete. 2024-03-28T18:00:20.635 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T18:00:20.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.856+0000 7faa9de00640 1 Processor -- start 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.857+0000 7faa9de00640 1 -- start start 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9de00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faa980b56f0 0x7faa980b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9de00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980b6e60 con 0x7faa9815dd60 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9de00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980b7100 con 0x7faa980b7ec0 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9de00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa980b6c30 con 0x7faa980b56f0 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9ca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faa9815dd60 0x7faa980b94b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60922/0 (socket says 172.21.15.62:60922) 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.858+0000 7faa9ca00640 1 -- 172.21.15.62:0/3803940624 learned_addr learned my addr 172.21.15.62:0/3803940624 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97e00640 1 --2- 172.21.15.62:0/3803940624 >> v2:172.21.15.81:3300/0 conn(0x7faa980b56f0 0x7faa980b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75932689 0 0) 0x7faa980b6e60 con 0x7faa9815dd60 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa80003520 con 0x7faa9815dd60 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1660680635 0 0) 0x7faa80003520 con 0x7faa9815dd60 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 >> v2:172.21.15.81:3300/0 conn(0x7faa980b56f0 msgr2=0x7faa980b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 --2- 172.21.15.62:0/3803940624 >> v2:172.21.15.81:3300/0 conn(0x7faa980b56f0 0x7faa980b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 >> v1:172.21.15.81:6789/0 conn(0x7faa980b7ec0 legacy=0x7faa980b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980b73e0 con 0x7faa9815dd60 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7faa8c001670 con 0x7faa9815dd60 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 >> v1:172.21.15.62:6789/0 conn(0x7faa9815dd60 legacy=0x7faa980b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8000c040 con 0x7faa800039b0 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8000c1f0 con 0x7faa80004800 2024-03-28T18:00:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.859+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8000c3f0 con 0x7faa800051c0 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa97e00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9d400640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 -- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800051c0 msgr2=0x7faa8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 -- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa80004800 msgr2=0x7faa80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9ca00640 1 -- 172.21.15.62:0/3803940624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80003520 con 0x7faa800039b0 2024-03-28T18:00:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.860+0000 7faa9d400640 1 -- 172.21.15.62:0/3803940624 reap_dead start 2024-03-28T18:00:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.861+0000 7faa9ca00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c240 tx=0x7faa80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c152bd4b38e7218 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c007cf0 con 0x7faa800039b0 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c030070 con 0x7faa800039b0 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa97400640 1 -- 172.21.15.62:0/3803940624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c005d70 con 0x7faa800039b0 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 -- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 msgr2=0x7faa80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c240 tx=0x7faa80005830 comp rx=0 tx=0).stop 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 -- 172.21.15.62:0/3803940624 shutdown_connections 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 --2- 172.21.15.62:0/3803940624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 -- 172.21.15.62:0/3803940624 >> 172.21.15.62:0/3803940624 conn(0x7faa98071ae0 msgr2=0x7faa98071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.862+0000 7faa9de00640 1 -- 172.21.15.62:0/3803940624 shutdown_connections 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.863+0000 7faa9de00640 1 -- 172.21.15.62:0/3803940624 wait complete. 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.864+0000 7faa9de00640 1 Processor -- start 2024-03-28T18:00:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.864+0000 7faa9de00640 1 -- start start 2024-03-28T18:00:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 0x7faa9814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa980cc6e0 0x7faa980d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815e140 con 0x7faa9815b4b0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980b79a0 con 0x7faa980cc6e0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.865+0000 7faa9de00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980b6370 con 0x7faa800039b0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42630/0 (socket says 172.21.15.62:42630) 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 -- 172.21.15.62:0/1591353844 learned_addr learned my addr 172.21.15.62:0/1591353844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa9ca00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 0x7faa9814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 -- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 msgr2=0x7faa9814e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 0x7faa9814e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 -- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa980cc6e0 msgr2=0x7faa980d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa9d400640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa980cc6e0 0x7faa980d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa980cc6e0 0x7faa980d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa97e00640 1 -- 172.21.15.62:0/1591353844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9814e5f0 con 0x7faa9815b4b0 2024-03-28T18:00:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.866+0000 7faa9ca00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 0x7faa9814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa97e00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faa840116d0 tx=0x7faa84011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400f9c0 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8400fb60 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84015460 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa980cb640 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.867+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa980cbc00 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.868+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa98072b90 con 0x7faa9815b4b0 2024-03-28T18:00:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.869+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faa84015600 con 0x7faa9815b4b0 2024-03-28T18:00:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.870+0000 7faa94c00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 0x7faa7806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.870+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa8408b440 con 0x7faa9815b4b0 2024-03-28T18:00:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.871+0000 7faa9ca00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 0x7faa7806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.872+0000 7faa9ca00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 0x7faa7806e0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa8c034000 tx=0x7faa8c039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:20.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:20.875+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7faa9814e5f0 con 0x7faa9815b4b0 2024-03-28T18:00:21.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.188+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa980d1840 con 0x7faa9815b4b0 2024-03-28T18:00:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.190+0000 7faa94c00640 1 -- 172.21.15.62:0/1591353844 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4977 (secure 0 0 0) 0x7faa84019030 con 0x7faa9815b4b0 2024-03-28T18:00:21.190 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:21.190 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3964298394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":3964298394},{"type":"v1","addr":"172.21.15.81:6837","nonce":3964298394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4151204969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4151204969},{"type":"v1","addr":"172.21.15.81:6836","nonce":4151204969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2711505092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2711505092},{"type":"v1","addr":"172.21.15.62:6835","nonce":2711505092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T18:00:17.037477+0000","modified":"2024-03-28T18:00:20.154562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6834/1272559486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1272559486},{"type":"v1","addr":"172.21.15.62:6834","nonce":1272559486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":2}]} 2024-03-28T18:00:21.190 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-03-28T18:00:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 msgr2=0x7faa7806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 0x7faa7806e0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa8c034000 tx=0x7faa8c039040 comp rx=0 tx=0).stop 2024-03-28T18:00:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 msgr2=0x7faa980cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faa840116d0 tx=0x7faa84011b90 comp rx=0 tx=0).stop 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 shutdown_connections 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7faa7806bc00 0x7faa7806e0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa980cc6e0 0x7faa980d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa9815b4b0 0x7faa980cad40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 --2- 172.21.15.62:0/1591353844 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7faa800039b0 0x7faa9814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 >> 172.21.15.62:0/1591353844 conn(0x7faa98071ae0 msgr2=0x7faa980d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 shutdown_connections 2024-03-28T18:00:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.192+0000 7faa9de00640 1 -- 172.21.15.62:0/1591353844 wait complete. 2024-03-28T18:00:21.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:00:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.415+0000 7f387d000640 1 Processor -- start 2024-03-28T18:00:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.416+0000 7f387d000640 1 -- start start 2024-03-28T18:00:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.417+0000 7f387d000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 0x7f387815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.417+0000 7f387d000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.417+0000 7f387d000640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38780cc880 con 0x7f387815e960 2024-03-28T18:00:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.417+0000 7f387d000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38781617c0 con 0x7f387815e040 2024-03-28T18:00:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.417+0000 7f387d000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387815f610 con 0x7f38781608c0 2024-03-28T18:00:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 0x7f387815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3301/0 says I am v2:172.21.15.62:56878/0 (socket says 172.21.15.62:56878) 2024-03-28T18:00:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 -- 172.21.15.62:0/289022972 learned_addr learned my addr 172.21.15.62:0/289022972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877400640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 0x7f387815e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36558/0 (socket says 172.21.15.62:36558) 2024-03-28T18:00:21.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 -- 172.21.15.62:0/289022972 >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 msgr2=0x7f387815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:21.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 --2- 172.21.15.62:0/289022972 >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 0x7f387815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.418+0000 7f3877e00640 1 -- 172.21.15.62:0/289022972 >> v1:172.21.15.81:6789/0 conn(0x7f387815e960 legacy=0x7f387815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.419+0000 7f3877e00640 1 -- 172.21.15.62:0/289022972 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780ccb20 con 0x7f38781608c0 2024-03-28T18:00:21.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.421+0000 7f3877e00640 1 --2- 172.21.15.62:0/289022972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3870009870 tx=0x7f387002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee06bc644092ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.421+0000 7f3876000640 1 -- 172.21.15.62:0/289022972 <== mon.2 v2:172.21.15.81:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870034070 con 0x7f38781608c0 2024-03-28T18:00:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.421+0000 7f3876000640 1 -- 172.21.15.62:0/289022972 <== mon.2 v2:172.21.15.81:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870030840 con 0x7f38781608c0 2024-03-28T18:00:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.421+0000 7f3876000640 1 -- 172.21.15.62:0/289022972 <== mon.2 v2:172.21.15.81:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870030c10 con 0x7f38781608c0 2024-03-28T18:00:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 -- 172.21.15.62:0/289022972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 msgr2=0x7f387806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 --2- 172.21.15.62:0/289022972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3870009870 tx=0x7f387002b980 comp rx=0 tx=0).stop 2024-03-28T18:00:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 -- 172.21.15.62:0/289022972 shutdown_connections 2024-03-28T18:00:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 --2- 172.21.15.62:0/289022972 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f38781608c0 0x7f387806d140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 --2- 172.21.15.62:0/289022972 >> v2:172.21.15.81:3300/0 conn(0x7f387815e040 0x7f387815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 -- 172.21.15.62:0/289022972 >> 172.21.15.62:0/289022972 conn(0x7f38780b1c70 msgr2=0x7f38780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 -- 172.21.15.62:0/289022972 shutdown_connections 2024-03-28T18:00:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.422+0000 7f387d000640 1 -- 172.21.15.62:0/289022972 wait complete. 2024-03-28T18:00:21.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.424+0000 7f387d000640 1 Processor -- start 2024-03-28T18:00:21.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.424+0000 7f387d000640 1 -- start start 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 0x7f38780c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38781608c0 0x7f38780cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387806d680 con 0x7f387815e040 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780cd0e0 con 0x7f38781608c0 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f387d000640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387806d800 con 0x7f387815e960 2024-03-28T18:00:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f3877400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f3877400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42644/0 (socket says 172.21.15.62:42644) 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.425+0000 7f3877400640 1 -- 172.21.15.62:0/2842780544 learned_addr learned my addr 172.21.15.62:0/2842780544 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3876a00640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 0x7f38780c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877e00640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38781608c0 0x7f38780cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 -- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 msgr2=0x7f38780c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 0x7f38780c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 -- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38781608c0 msgr2=0x7f38780cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38781608c0 0x7f38780cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 -- 172.21.15.62:0/2842780544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780c8cb0 con 0x7f387815e040 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3876a00640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 0x7f38780c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.426+0000 7f3877400640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3868009db0 tx=0x7f386800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.427+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868018020 con 0x7f387815e040 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.427+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868014480 con 0x7f387815e040 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.427+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38780c8f90 con 0x7f387815e040 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.427+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868014850 con 0x7f387815e040 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.427+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38780c9c90 con 0x7f387815e040 2024-03-28T18:00:21.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.428+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38780b1a20 con 0x7f387815e040 2024-03-28T18:00:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.429+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3868005d10 con 0x7f387815e040 2024-03-28T18:00:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.430+0000 7f386f400640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 0x7f385806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.430+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f386808b6a0 con 0x7f387815e040 2024-03-28T18:00:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.430+0000 7f3876a00640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 0x7f385806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.432+0000 7f3876a00640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 0x7f385806e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38780cfba0 tx=0x7f3864009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.439+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f38780b1a20 con 0x7f387815e040 2024-03-28T18:00:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.682+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f387804f4e0 con 0x7f387815e040 2024-03-28T18:00:21.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.684+0000 7f386f400640 1 -- 172.21.15.62:0/2842780544 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f3868022070 con 0x7f387815e040 2024-03-28T18:00:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:21.685 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","created":"2024-03-28T17:58:29.006818+0000","modified":"2024-03-28T18:00:18.552374+0000","last_up_change":"2024-03-28T17:58:41.570267+0000","last_in_change":"2024-03-28T17:58:32.409101+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:42.378514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:00:15.184773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T18:00:16.044475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33eb1999-42d3-4888-8067-853d79e67f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6809","nonce":2494694084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6811","nonce":2494694084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6815","nonce":2494694084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2494694084},{"type":"v1","addr":"172.21.15.62:6813","nonce":2494694084}]},"public_addr":"172.21.15.62:6809/2494694084","cluster_addr":"172.21.15.62:6811/2494694084","heartbeat_back_addr":"172.21.15.62:6815/2494694084","heartbeat_front_addr":"172.21.15.62:6813/2494694084","state":["exists","up"]},{"osd":1,"uuid":"e7677661-2399-419b-af43-51c30814f65f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6801","nonce":2984117190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6803","nonce":2984117190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6807","nonce":2984117190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2984117190},{"type":"v1","addr":"172.21.15.62:6805","nonce":2984117190}]},"public_addr":"172.21.15.62:6801/2984117190","cluster_addr":"172.21.15.62:6803/2984117190","heartbeat_back_addr":"172.21.15.62:6807/2984117190","heartbeat_front_addr":"172.21.15.62:6805/2984117190","state":["exists","up"]},{"osd":2,"uuid":"584b7a16-2729-4cf0-a52a-2384928197c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6817","nonce":2545705351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6819","nonce":2545705351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6823","nonce":2545705351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2545705351},{"type":"v1","addr":"172.21.15.62:6821","nonce":2545705351}]},"public_addr":"172.21.15.62:6817/2545705351","cluster_addr":"172.21.15.62:6819/2545705351","heartbeat_back_addr":"172.21.15.62:6823/2545705351","heartbeat_front_addr":"172.21.15.62:6821/2545705351","state":["exists","up"]},{"osd":3,"uuid":"a08a6c7d-b71a-4329-8849-5add820b36ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6825","nonce":1755400428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6827","nonce":1755400428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6831","nonce":1755400428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1755400428},{"type":"v1","addr":"172.21.15.62:6829","nonce":1755400428}]},"public_addr":"172.21.15.62:6825/1755400428","cluster_addr":"172.21.15.62:6827/1755400428","heartbeat_back_addr":"172.21.15.62:6831/1755400428","heartbeat_front_addr":"172.21.15.62:6829/1755400428","state":["exists","up"]},{"osd":4,"uuid":"9a37caf2-9668-48c1-a579-29fa3626208f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6825","nonce":2310676822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6827","nonce":2310676822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6831","nonce":2310676822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2310676822},{"type":"v1","addr":"172.21.15.81:6829","nonce":2310676822}]},"public_addr":"172.21.15.81:6825/2310676822","cluster_addr":"172.21.15.81:6827/2310676822","heartbeat_back_addr":"172.21.15.81:6831/2310676822","heartbeat_front_addr":"172.21.15.81:6829/2310676822","state":["exists","up"]},{"osd":5,"uuid":"78fef44b-4df0-406d-8b8f-026bc08bc74c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6801","nonce":619433227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6803","nonce":619433227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6807","nonce":619433227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":619433227},{"type":"v1","addr":"172.21.15.81:6805","nonce":619433227}]},"public_addr":"172.21.15.81:6801/619433227","cluster_addr":"172.21.15.81:6803/619433227","heartbeat_back_addr":"172.21.15.81:6807/619433227","heartbeat_front_addr":"172.21.15.81:6805/619433227","state":["exists","up"]},{"osd":6,"uuid":"a9a414ab-f06e-4bbb-8f8b-2939857fca35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6809","nonce":1688303827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6811","nonce":1688303827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6815","nonce":1688303827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1688303827},{"type":"v1","addr":"172.21.15.81:6813","nonce":1688303827}]},"public_addr":"172.21.15.81:6809/1688303827","cluster_addr":"172.21.15.81:6811/1688303827","heartbeat_back_addr":"172.21.15.81:6815/1688303827","heartbeat_front_addr":"172.21.15.81:6813/1688303827","state":["exists","up"]},{"osd":7,"uuid":"67867055-2853-4975-895d-2f444fc83ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6817","nonce":2473781039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6819","nonce":2473781039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6823","nonce":2473781039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2473781039},{"type":"v1","addr":"172.21.15.81:6821","nonce":2473781039}]},"public_addr":"172.21.15.81:6817/2473781039","cluster_addr":"172.21.15.81:6819/2473781039","heartbeat_back_addr":"172.21.15.81:6823/2473781039","heartbeat_front_addr":"172.21.15.81:6821/2473781039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.743923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.952774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.536688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.996344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.770953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.531170+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.719539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:37.681819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 msgr2=0x7f385806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 0x7f385806e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38780cfba0 tx=0x7f3864009040 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 msgr2=0x7f38781516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3868009db0 tx=0x7f386800c490 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 shutdown_connections 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f385806bbd0 0x7f385806e090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38781608c0 0x7f38780cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f387815e960 0x7f38780c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.687+0000 7f387d000640 1 --2- 172.21.15.62:0/2842780544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387815e040 0x7f38781516f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.688+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 >> 172.21.15.62:0/2842780544 conn(0x7f38780b1c70 msgr2=0x7f38780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.688+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 shutdown_connections 2024-03-28T18:00:21.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.688+0000 7f387d000640 1 -- 172.21.15.62:0/2842780544 wait complete. 2024-03-28T18:00:21.708 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T18:00:21.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:00:21.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.942+0000 7f2937200640 1 Processor -- start 2024-03-28T18:00:21.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.942+0000 7f2937200640 1 -- start start 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f293815dd60 0x7f29380b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f29380b99f0 0x7f29380b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f29380b5c30 0x7f2938161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380b60a0 con 0x7f29380b5c30 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380b6c30 con 0x7f293815dd60 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.943+0000 7f2937200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380b6e60 con 0x7f29380b99f0 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f29380b5c30 0x7f2938161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f29380b5c30 0x7f2938161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42658/0 (socket says 172.21.15.62:42658) 2024-03-28T18:00:21.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 -- 172.21.15.62:0/3370138422 learned_addr learned my addr 172.21.15.62:0/3370138422 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 -- 172.21.15.62:0/3370138422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f29380b99f0 msgr2=0x7f29380b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 --2- 172.21.15.62:0/3370138422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f29380b99f0 0x7f29380b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 -- 172.21.15.62:0/3370138422 >> v2:172.21.15.81:3300/0 conn(0x7f293815dd60 msgr2=0x7f29380b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 --2- 172.21.15.62:0/3370138422 >> v2:172.21.15.81:3300/0 conn(0x7f293815dd60 0x7f29380b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.944+0000 7f2936800640 1 -- 172.21.15.62:0/3370138422 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380b7100 con 0x7f29380b5c30 2024-03-28T18:00:21.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.945+0000 7f2936800640 1 --2- 172.21.15.62:0/3370138422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b5c30 0x7f2938161950 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f29380d1570 tx=0x7f293002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b69fe9fa9efd806 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2935400640 1 -- 172.21.15.62:0/3370138422 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293002cb00 con 0x7f29380b5c30 2024-03-28T18:00:21.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2935400640 1 -- 172.21.15.62:0/3370138422 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293002cca0 con 0x7f29380b5c30 2024-03-28T18:00:21.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2935400640 1 -- 172.21.15.62:0/3370138422 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930034610 con 0x7f29380b5c30 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 -- 172.21.15.62:0/3370138422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b5c30 msgr2=0x7f2938161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 --2- 172.21.15.62:0/3370138422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b5c30 0x7f2938161950 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f29380d1570 tx=0x7f293002bf60 comp rx=0 tx=0).stop 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 -- 172.21.15.62:0/3370138422 shutdown_connections 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 --2- 172.21.15.62:0/3370138422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b5c30 0x7f2938161950 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 --2- 172.21.15.62:0/3370138422 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f29380b99f0 0x7f29380b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 --2- 172.21.15.62:0/3370138422 >> v2:172.21.15.81:3300/0 conn(0x7f293815dd60 0x7f29380b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 -- 172.21.15.62:0/3370138422 >> 172.21.15.62:0/3370138422 conn(0x7f2938071ae0 msgr2=0x7f2938071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 -- 172.21.15.62:0/3370138422 shutdown_connections 2024-03-28T18:00:21.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.946+0000 7f2937200640 1 -- 172.21.15.62:0/3370138422 wait complete. 2024-03-28T18:00:21.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.948+0000 7f2937200640 1 Processor -- start 2024-03-28T18:00:21.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.948+0000 7f2937200640 1 -- start start 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29380b5c30 0x7f293814c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f293815dd60 0x7f293815fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380b79a0 con 0x7f29380b99f0 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380b76c0 con 0x7f29380b5c30 2024-03-28T18:00:21.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f2937200640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293815e140 con 0x7f293815dd60 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f292fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f292fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42674/0 (socket says 172.21.15.62:42674) 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.949+0000 7f292fe00640 1 -- 172.21.15.62:0/1853473870 learned_addr learned my addr 172.21.15.62:0/1853473870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f2936800640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f293815dd60 0x7f293815fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f2935e00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29380b5c30 0x7f293814c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 -- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f293815dd60 msgr2=0x7f293815fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f293815dd60 0x7f293815fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 -- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29380b5c30 msgr2=0x7f293814c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29380b5c30 0x7f293814c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 -- 172.21.15.62:0/1853473870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380c8190 con 0x7f29380b99f0 2024-03-28T18:00:21.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.950+0000 7f292fe00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f292000b4d0 tx=0x7f292000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.951+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920014070 con 0x7f29380b99f0 2024-03-28T18:00:21.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.951+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2920010870 con 0x7f29380b99f0 2024-03-28T18:00:21.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.951+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920010c60 con 0x7f29380b99f0 2024-03-28T18:00:21.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.951+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29380c8470 con 0x7f29380b99f0 2024-03-28T18:00:21.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.951+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29380c8a30 con 0x7f29380b99f0 2024-03-28T18:00:21.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.952+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938072b90 con 0x7f29380b99f0 2024-03-28T18:00:21.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.954+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2920017070 con 0x7f29380b99f0 2024-03-28T18:00:21.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.955+0000 7f292e000640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 0x7f291406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:21.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.956+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f292008a7a0 con 0x7f29380b99f0 2024-03-28T18:00:21.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.956+0000 7f2935e00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 0x7f291406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:21.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.957+0000 7f2935e00640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 0x7f291406e090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29381f7650 tx=0x7f2928008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:21.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:21.961+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f2920059320 con 0x7f29380b99f0 2024-03-28T18:00:22.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.162+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f293804f4e0 con 0x7f291406bbd0 2024-03-28T18:00:22.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.183+0000 7f292e000640 1 -- 172.21.15.62:0/1853473870 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f293804f4e0 con 0x7f291406bbd0 2024-03-28T18:00:22.187 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:22.188 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 msgr2=0x7f291406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 0x7f291406e090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29381f7650 tx=0x7f2928008040 comp rx=0 tx=0).stop 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 msgr2=0x7f29380c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f292000b4d0 tx=0x7f292000b990 comp rx=0 tx=0).stop 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 shutdown_connections 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f291406bbd0 0x7f291406e090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f293815dd60 0x7f293815fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29380b99f0 0x7f29380c7740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 --2- 172.21.15.62:0/1853473870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29380b5c30 0x7f293814c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 >> 172.21.15.62:0/1853473870 conn(0x7f2938071ae0 msgr2=0x7f29380b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:22.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 shutdown_connections 2024-03-28T18:00:22.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.188+0000 7f2937200640 1 -- 172.21.15.62:0/1853473870 wait complete. 2024-03-28T18:00:22.210 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-28T18:00:20.335347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3960832,"data_stored":2242797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.798260"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569583+0000","last_change":"2024-03-28T18:00:16.569583+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569583+0000","last_undegraded":"2024-03-28T18:00:16.569583+0000","last_fullsized":"2024-03-28T18:00:16.569583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579654+0000","last_change":"2024-03-28T18:00:16.579654+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579654+0000","last_undegraded":"2024-03-28T18:00:16.579654+0000","last_fullsized":"2024-03-28T18:00:16.579654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561545+0000","last_change":"2024-03-28T18:00:16.561545+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561545+0000","last_undegraded":"2024-03-28T18:00:16.561545+0000","last_fullsized":"2024-03-28T18:00:16.561545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571390+0000","last_change":"2024-03-28T18:00:16.571390+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571390+0000","last_undegraded":"2024-03-28T18:00:16.571390+0000","last_fullsized":"2024-03-28T18:00:16.571390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557319+0000","last_change":"2024-03-28T18:00:16.557319+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557319+0000","last_undegraded":"2024-03-28T18:00:16.557319+0000","last_fullsized":"2024-03-28T18:00:16.557319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.580779+0000","last_change":"2024-03-28T18:00:16.580779+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.580779+0000","last_undegraded":"2024-03-28T18:00:16.580779+0000","last_fullsized":"2024-03-28T18:00:16.580779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568442+0000","last_change":"2024-03-28T18:00:16.568442+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568442+0000","last_undegraded":"2024-03-28T18:00:16.568442+0000","last_fullsized":"2024-03-28T18:00:16.568442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575897+0000","last_change":"2024-03-28T18:00:16.575897+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575897+0000","last_undegraded":"2024-03-28T18:00:16.575897+0000","last_fullsized":"2024-03-28T18:00:16.575897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576562+0000","last_change":"2024-03-28T18:00:16.576562+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576562+0000","last_undegraded":"2024-03-28T18:00:16.576562+0000","last_fullsized":"2024-03-28T18:00:16.576562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568412+0000","last_change":"2024-03-28T18:00:16.568412+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568412+0000","last_undegraded":"2024-03-28T18:00:16.568412+0000","last_fullsized":"2024-03-28T18:00:16.568412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569231+0000","last_change":"2024-03-28T18:00:16.569231+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569231+0000","last_undegraded":"2024-03-28T18:00:16.569231+0000","last_fullsized":"2024-03-28T18:00:16.569231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559639+0000","last_change":"2024-03-28T18:00:16.559639+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559639+0000","last_undegraded":"2024-03-28T18:00:16.559639+0000","last_fullsized":"2024-03-28T18:00:16.559639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556645+0000","last_change":"2024-03-28T18:00:16.556645+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556645+0000","last_undegraded":"2024-03-28T18:00:16.556645+0000","last_fullsized":"2024-03-28T18:00:16.556645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572147+0000","last_change":"2024-03-28T18:00:16.572147+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572147+0000","last_undegraded":"2024-03-28T18:00:16.572147+0000","last_fullsized":"2024-03-28T18:00:16.572147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557955+0000","last_change":"2024-03-28T18:00:16.557955+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557955+0000","last_undegraded":"2024-03-28T18:00:16.557955+0000","last_fullsized":"2024-03-28T18:00:16.557955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567619+0000","last_change":"2024-03-28T18:00:16.567619+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567619+0000","last_undegraded":"2024-03-28T18:00:16.567619+0000","last_fullsized":"2024-03-28T18:00:16.567619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567977+0000","last_change":"2024-03-28T18:00:16.567977+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567977+0000","last_undegraded":"2024-03-28T18:00:16.567977+0000","last_fullsized":"2024-03-28T18:00:16.567977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.577391+0000","last_change":"2024-03-28T18:00:16.577391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.577391+0000","last_undegraded":"2024-03-28T18:00:16.577391+0000","last_fullsized":"2024-03-28T18:00:16.577391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573104+0000","last_change":"2024-03-28T18:00:16.573104+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573104+0000","last_undegraded":"2024-03-28T18:00:16.573104+0000","last_fullsized":"2024-03-28T18:00:16.573104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555854+0000","last_change":"2024-03-28T18:00:16.555854+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555854+0000","last_undegraded":"2024-03-28T18:00:16.555854+0000","last_fullsized":"2024-03-28T18:00:16.555854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576380+0000","last_change":"2024-03-28T18:00:16.576380+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576380+0000","last_undegraded":"2024-03-28T18:00:16.576380+0000","last_fullsized":"2024-03-28T18:00:16.576380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571362+0000","last_change":"2024-03-28T18:00:16.571362+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571362+0000","last_undegraded":"2024-03-28T18:00:16.571362+0000","last_fullsized":"2024-03-28T18:00:16.571362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561762+0000","last_change":"2024-03-28T18:00:16.561762+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561762+0000","last_undegraded":"2024-03-28T18:00:16.561762+0000","last_fullsized":"2024-03-28T18:00:16.561762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558028+0000","last_change":"2024-03-28T18:00:16.558028+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558028+0000","last_undegraded":"2024-03-28T18:00:16.558028+0000","last_fullsized":"2024-03-28T18:00:16.558028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569204+0000","last_change":"2024-03-28T18:00:16.569204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569204+0000","last_undegraded":"2024-03-28T18:00:16.569204+0000","last_fullsized":"2024-03-28T18:00:16.569204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576919+0000","last_change":"2024-03-28T18:00:16.576919+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576919+0000","last_undegraded":"2024-03-28T18:00:16.576919+0000","last_fullsized":"2024-03-28T18:00:16.576919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566641+0000","last_change":"2024-03-28T18:00:16.566641+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566641+0000","last_undegraded":"2024-03-28T18:00:16.566641+0000","last_fullsized":"2024-03-28T18:00:16.566641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558269+0000","last_change":"2024-03-28T18:00:16.558269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558269+0000","last_undegraded":"2024-03-28T18:00:16.558269+0000","last_fullsized":"2024-03-28T18:00:16.558269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575581+0000","last_change":"2024-03-28T18:00:16.575581+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575581+0000","last_undegraded":"2024-03-28T18:00:16.575581+0000","last_fullsized":"2024-03-28T18:00:16.575581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572204+0000","last_change":"2024-03-28T18:00:16.572204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572204+0000","last_undegraded":"2024-03-28T18:00:16.572204+0000","last_fullsized":"2024-03-28T18:00:16.572204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571667+0000","last_change":"2024-03-28T18:00:16.571667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571667+0000","last_undegraded":"2024-03-28T18:00:16.571667+0000","last_fullsized":"2024-03-28T18:00:16.571667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574076+0000","last_change":"2024-03-28T18:00:16.574076+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574076+0000","last_undegraded":"2024-03-28T18:00:16.574076+0000","last_fullsized":"2024-03-28T18:00:16.574076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562017+0000","last_change":"2024-03-28T18:00:16.562017+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562017+0000","last_undegraded":"2024-03-28T18:00:16.562017+0000","last_fullsized":"2024-03-28T18:00:16.562017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588360+0000","last_change":"2024-03-28T18:00:16.586041+0000","last_active":"2024-03-28T18:00:16.588360+0000","last_peered":"2024-03-28T18:00:16.588360+0000","last_clean":"2024-03-28T18:00:16.588360+0000","last_became_active":"2024-03-28T18:00:16.585654+0000","last_became_peered":"2024-03-28T18:00:16.585654+0000","last_unstale":"2024-03-28T18:00:16.588360+0000","last_undegraded":"2024-03-28T18:00:16.588360+0000","last_fullsized":"2024-03-28T18:00:16.588360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567054+0000","last_change":"2024-03-28T18:00:16.567054+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567054+0000","last_undegraded":"2024-03-28T18:00:16.567054+0000","last_fullsized":"2024-03-28T18:00:16.567054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586926+0000","last_change":"2024-03-28T18:00:16.582651+0000","last_active":"2024-03-28T18:00:16.586926+0000","last_peered":"2024-03-28T18:00:16.586926+0000","last_clean":"2024-03-28T18:00:16.586926+0000","last_became_active":"2024-03-28T18:00:16.582248+0000","last_became_peered":"2024-03-28T18:00:16.582248+0000","last_unstale":"2024-03-28T18:00:16.586926+0000","last_undegraded":"2024-03-28T18:00:16.586926+0000","last_fullsized":"2024-03-28T18:00:16.586926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571368+0000","last_change":"2024-03-28T18:00:16.571368+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571368+0000","last_undegraded":"2024-03-28T18:00:16.571368+0000","last_fullsized":"2024-03-28T18:00:16.571368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592321+0000","last_change":"2024-03-28T18:00:16.588264+0000","last_active":"2024-03-28T18:00:16.592321+0000","last_peered":"2024-03-28T18:00:16.592321+0000","last_clean":"2024-03-28T18:00:16.592321+0000","last_became_active":"2024-03-28T18:00:16.587925+0000","last_became_peered":"2024-03-28T18:00:16.587925+0000","last_unstale":"2024-03-28T18:00:16.592321+0000","last_undegraded":"2024-03-28T18:00:16.592321+0000","last_fullsized":"2024-03-28T18:00:16.592321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565721+0000","last_change":"2024-03-28T18:00:16.565721+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565721+0000","last_undegraded":"2024-03-28T18:00:16.565721+0000","last_fullsized":"2024-03-28T18:00:16.565721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579273+0000","last_change":"2024-03-28T18:00:16.577746+0000","last_active":"2024-03-28T18:00:16.579273+0000","last_peered":"2024-03-28T18:00:16.579273+0000","last_clean":"2024-03-28T18:00:16.579273+0000","last_became_active":"2024-03-28T18:00:16.577340+0000","last_became_peered":"2024-03-28T18:00:16.577340+0000","last_unstale":"2024-03-28T18:00:16.579273+0000","last_undegraded":"2024-03-28T18:00:16.579273+0000","last_fullsized":"2024-03-28T18:00:16.579273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570787+0000","last_change":"2024-03-28T18:00:16.570787+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570787+0000","last_undegraded":"2024-03-28T18:00:16.570787+0000","last_fullsized":"2024-03-28T18:00:16.570787+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593331+0000","last_change":"2024-03-28T18:00:16.591523+0000","last_active":"2024-03-28T18:00:16.593331+0000","last_peered":"2024-03-28T18:00:16.593331+0000","last_clean":"2024-03-28T18:00:16.593331+0000","last_became_active":"2024-03-28T18:00:16.591203+0000","last_became_peered":"2024-03-28T18:00:16.591203+0000","last_unstale":"2024-03-28T18:00:16.593331+0000","last_undegraded":"2024-03-28T18:00:16.593331+0000","last_fullsized":"2024-03-28T18:00:16.593331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:13.152406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559660+0000","last_change":"2024-03-28T18:00:16.559660+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559660+0000","last_undegraded":"2024-03-28T18:00:16.559660+0000","last_fullsized":"2024-03-28T18:00:16.559660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593355+0000","last_change":"2024-03-28T18:00:16.591608+0000","last_active":"2024-03-28T18:00:16.593355+0000","last_peered":"2024-03-28T18:00:16.593355+0000","last_clean":"2024-03-28T18:00:16.593355+0000","last_became_active":"2024-03-28T18:00:16.591289+0000","last_became_peered":"2024-03-28T18:00:16.591289+0000","last_unstale":"2024-03-28T18:00:16.593355+0000","last_undegraded":"2024-03-28T18:00:16.593355+0000","last_fullsized":"2024-03-28T18:00:16.593355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:23.345799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574763+0000","last_change":"2024-03-28T18:00:16.574763+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574763+0000","last_undegraded":"2024-03-28T18:00:16.574763+0000","last_fullsized":"2024-03-28T18:00:16.574763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589450+0000","last_change":"2024-03-28T18:00:16.581766+0000","last_active":"2024-03-28T18:00:16.589450+0000","last_peered":"2024-03-28T18:00:16.589450+0000","last_clean":"2024-03-28T18:00:16.589450+0000","last_became_active":"2024-03-28T18:00:16.581506+0000","last_became_peered":"2024-03-28T18:00:16.581506+0000","last_unstale":"2024-03-28T18:00:16.589450+0000","last_undegraded":"2024-03-28T18:00:16.589450+0000","last_fullsized":"2024-03-28T18:00:16.589450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.564269+0000","last_change":"2024-03-28T18:00:16.564269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.564269+0000","last_undegraded":"2024-03-28T18:00:16.564269+0000","last_fullsized":"2024-03-28T18:00:16.564269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590746+0000","last_change":"2024-03-28T18:00:16.589309+0000","last_active":"2024-03-28T18:00:16.590746+0000","last_peered":"2024-03-28T18:00:16.590746+0000","last_clean":"2024-03-28T18:00:16.590746+0000","last_became_active":"2024-03-28T18:00:16.588995+0000","last_became_peered":"2024-03-28T18:00:16.588995+0000","last_unstale":"2024-03-28T18:00:16.590746+0000","last_undegraded":"2024-03-28T18:00:16.590746+0000","last_fullsized":"2024-03-28T18:00:16.590746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:10.952085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560002+0000","last_change":"2024-03-28T18:00:16.560002+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560002+0000","last_undegraded":"2024-03-28T18:00:16.560002+0000","last_fullsized":"2024-03-28T18:00:16.560002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592334+0000","last_change":"2024-03-28T18:00:16.588159+0000","last_active":"2024-03-28T18:00:16.592334+0000","last_peered":"2024-03-28T18:00:16.592334+0000","last_clean":"2024-03-28T18:00:16.592334+0000","last_became_active":"2024-03-28T18:00:16.587891+0000","last_became_peered":"2024-03-28T18:00:16.587891+0000","last_unstale":"2024-03-28T18:00:16.592334+0000","last_undegraded":"2024-03-28T18:00:16.592334+0000","last_fullsized":"2024-03-28T18:00:16.592334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574440+0000","last_change":"2024-03-28T18:00:16.574440+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574440+0000","last_undegraded":"2024-03-28T18:00:16.574440+0000","last_fullsized":"2024-03-28T18:00:16.574440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592843+0000","last_change":"2024-03-28T18:00:16.590209+0000","last_active":"2024-03-28T18:00:16.592843+0000","last_peered":"2024-03-28T18:00:16.592843+0000","last_clean":"2024-03-28T18:00:16.592843+0000","last_became_active":"2024-03-28T18:00:16.589874+0000","last_became_peered":"2024-03-28T18:00:16.589874+0000","last_unstale":"2024-03-28T18:00:16.592843+0000","last_undegraded":"2024-03-28T18:00:16.592843+0000","last_fullsized":"2024-03-28T18:00:16.592843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573906+0000","last_change":"2024-03-28T18:00:16.573906+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573906+0000","last_undegraded":"2024-03-28T18:00:16.573906+0000","last_fullsized":"2024-03-28T18:00:16.573906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588152+0000","last_change":"2024-03-28T18:00:16.583925+0000","last_active":"2024-03-28T18:00:16.588152+0000","last_peered":"2024-03-28T18:00:16.588152+0000","last_clean":"2024-03-28T18:00:16.588152+0000","last_became_active":"2024-03-28T18:00:16.583635+0000","last_became_peered":"2024-03-28T18:00:16.583635+0000","last_unstale":"2024-03-28T18:00:16.588152+0000","last_undegraded":"2024-03-28T18:00:16.588152+0000","last_fullsized":"2024-03-28T18:00:16.588152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562694+0000","last_change":"2024-03-28T18:00:16.562694+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562694+0000","last_undegraded":"2024-03-28T18:00:16.562694+0000","last_fullsized":"2024-03-28T18:00:16.562694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.597940+0000","last_change":"2024-03-28T18:00:16.593104+0000","last_active":"2024-03-28T18:00:16.597940+0000","last_peered":"2024-03-28T18:00:16.597940+0000","last_clean":"2024-03-28T18:00:16.597940+0000","last_became_active":"2024-03-28T18:00:16.592771+0000","last_became_peered":"2024-03-28T18:00:16.592771+0000","last_unstale":"2024-03-28T18:00:16.597940+0000","last_undegraded":"2024-03-28T18:00:16.597940+0000","last_fullsized":"2024-03-28T18:00:16.597940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565690+0000","last_change":"2024-03-28T18:00:16.565690+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565690+0000","last_undegraded":"2024-03-28T18:00:16.565690+0000","last_fullsized":"2024-03-28T18:00:16.565690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589009+0000","last_change":"2024-03-28T18:00:16.584849+0000","last_active":"2024-03-28T18:00:16.589009+0000","last_peered":"2024-03-28T18:00:16.589009+0000","last_clean":"2024-03-28T18:00:16.589009+0000","last_became_active":"2024-03-28T18:00:16.584543+0000","last_became_peered":"2024-03-28T18:00:16.584543+0000","last_unstale":"2024-03-28T18:00:16.589009+0000","last_undegraded":"2024-03-28T18:00:16.589009+0000","last_fullsized":"2024-03-28T18:00:16.589009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:28:00.623248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579339+0000","last_change":"2024-03-28T18:00:16.579339+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579339+0000","last_undegraded":"2024-03-28T18:00:16.579339+0000","last_fullsized":"2024-03-28T18:00:16.579339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583382+0000","last_change":"2024-03-28T18:00:16.580448+0000","last_active":"2024-03-28T18:00:16.583382+0000","last_peered":"2024-03-28T18:00:16.583382+0000","last_clean":"2024-03-28T18:00:16.583382+0000","last_became_active":"2024-03-28T18:00:16.579745+0000","last_became_peered":"2024-03-28T18:00:16.579745+0000","last_unstale":"2024-03-28T18:00:16.583382+0000","last_undegraded":"2024-03-28T18:00:16.583382+0000","last_fullsized":"2024-03-28T18:00:16.583382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571904+0000","last_change":"2024-03-28T18:00:16.571904+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571904+0000","last_undegraded":"2024-03-28T18:00:16.571904+0000","last_fullsized":"2024-03-28T18:00:16.571904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592117+0000","last_change":"2024-03-28T18:00:16.587497+0000","last_active":"2024-03-28T18:00:16.592117+0000","last_peered":"2024-03-28T18:00:16.592117+0000","last_clean":"2024-03-28T18:00:16.592117+0000","last_became_active":"2024-03-28T18:00:16.586180+0000","last_became_peered":"2024-03-28T18:00:16.586180+0000","last_unstale":"2024-03-28T18:00:16.592117+0000","last_undegraded":"2024-03-28T18:00:16.592117+0000","last_fullsized":"2024-03-28T18:00:16.592117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566667+0000","last_change":"2024-03-28T18:00:16.566667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566667+0000","last_undegraded":"2024-03-28T18:00:16.566667+0000","last_fullsized":"2024-03-28T18:00:16.566667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589771+0000","last_change":"2024-03-28T18:00:16.581297+0000","last_active":"2024-03-28T18:00:16.589771+0000","last_peered":"2024-03-28T18:00:16.589771+0000","last_clean":"2024-03-28T18:00:16.589771+0000","last_became_active":"2024-03-28T18:00:16.580937+0000","last_became_peered":"2024-03-28T18:00:16.580937+0000","last_unstale":"2024-03-28T18:00:16.589771+0000","last_undegraded":"2024-03-28T18:00:16.589771+0000","last_fullsized":"2024-03-28T18:00:16.589771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571676+0000","last_change":"2024-03-28T18:00:16.571676+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571676+0000","last_undegraded":"2024-03-28T18:00:16.571676+0000","last_fullsized":"2024-03-28T18:00:16.571676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592286+0000","last_change":"2024-03-28T18:00:16.589262+0000","last_active":"2024-03-28T18:00:16.592286+0000","last_peered":"2024-03-28T18:00:16.592286+0000","last_clean":"2024-03-28T18:00:16.592286+0000","last_became_active":"2024-03-28T18:00:16.588786+0000","last_became_peered":"2024-03-28T18:00:16.588786+0000","last_unstale":"2024-03-28T18:00:16.592286+0000","last_undegraded":"2024-03-28T18:00:16.592286+0000","last_fullsized":"2024-03-28T18:00:16.592286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561624+0000","last_change":"2024-03-28T18:00:16.561624+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561624+0000","last_undegraded":"2024-03-28T18:00:16.561624+0000","last_fullsized":"2024-03-28T18:00:16.561624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589122+0000","last_change":"2024-03-28T18:00:16.587380+0000","last_active":"2024-03-28T18:00:16.589122+0000","last_peered":"2024-03-28T18:00:16.589122+0000","last_clean":"2024-03-28T18:00:16.589122+0000","last_became_active":"2024-03-28T18:00:16.586842+0000","last_became_peered":"2024-03-28T18:00:16.586842+0000","last_unstale":"2024-03-28T18:00:16.589122+0000","last_undegraded":"2024-03-28T18:00:16.589122+0000","last_fullsized":"2024-03-28T18:00:16.589122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.737137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570537+0000","last_change":"2024-03-28T18:00:16.570537+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570537+0000","last_undegraded":"2024-03-28T18:00:16.570537+0000","last_fullsized":"2024-03-28T18:00:16.570537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588036+0000","last_change":"2024-03-28T18:00:16.581961+0000","last_active":"2024-03-28T18:00:16.588036+0000","last_peered":"2024-03-28T18:00:16.588036+0000","last_clean":"2024-03-28T18:00:16.588036+0000","last_became_active":"2024-03-28T18:00:16.581694+0000","last_became_peered":"2024-03-28T18:00:16.581694+0000","last_unstale":"2024-03-28T18:00:16.588036+0000","last_undegraded":"2024-03-28T18:00:16.588036+0000","last_fullsized":"2024-03-28T18:00:16.588036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:58:38.567836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556811+0000","last_change":"2024-03-28T18:00:16.556811+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556811+0000","last_undegraded":"2024-03-28T18:00:16.556811+0000","last_fullsized":"2024-03-28T18:00:16.556811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.591372+0000","last_change":"2024-03-28T18:00:16.590452+0000","last_active":"2024-03-28T18:00:16.591372+0000","last_peered":"2024-03-28T18:00:16.591372+0000","last_clean":"2024-03-28T18:00:16.591372+0000","last_became_active":"2024-03-28T18:00:16.590148+0000","last_became_peered":"2024-03-28T18:00:16.590148+0000","last_unstale":"2024-03-28T18:00:16.591372+0000","last_undegraded":"2024-03-28T18:00:16.591372+0000","last_fullsized":"2024-03-28T18:00:16.591372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:59.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555930+0000","last_change":"2024-03-28T18:00:16.555930+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555930+0000","last_undegraded":"2024-03-28T18:00:16.555930+0000","last_fullsized":"2024-03-28T18:00:16.555930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.578677+0000","last_change":"2024-03-28T18:00:16.574301+0000","last_active":"2024-03-28T18:00:16.578677+0000","last_peered":"2024-03-28T18:00:16.578677+0000","last_clean":"2024-03-28T18:00:16.578677+0000","last_became_active":"2024-03-28T18:00:16.573671+0000","last_became_peered":"2024-03-28T18:00:16.573671+0000","last_unstale":"2024-03-28T18:00:16.578677+0000","last_undegraded":"2024-03-28T18:00:16.578677+0000","last_fullsized":"2024-03-28T18:00:16.578677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555199+0000","last_change":"2024-03-28T18:00:16.555199+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555199+0000","last_undegraded":"2024-03-28T18:00:16.555199+0000","last_fullsized":"2024-03-28T18:00:16.555199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586129+0000","last_change":"2024-03-28T18:00:16.584421+0000","last_active":"2024-03-28T18:00:16.586129+0000","last_peered":"2024-03-28T18:00:16.586129+0000","last_clean":"2024-03-28T18:00:16.586129+0000","last_became_active":"2024-03-28T18:00:16.584113+0000","last_became_peered":"2024-03-28T18:00:16.584113+0000","last_unstale":"2024-03-28T18:00:16.586129+0000","last_undegraded":"2024-03-28T18:00:16.586129+0000","last_fullsized":"2024-03-28T18:00:16.586129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556334+0000","last_change":"2024-03-28T18:00:16.556334+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556334+0000","last_undegraded":"2024-03-28T18:00:16.556334+0000","last_fullsized":"2024-03-28T18:00:16.556334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.582176+0000","last_change":"2024-03-28T18:00:16.578260+0000","last_active":"2024-03-28T18:00:16.582176+0000","last_peered":"2024-03-28T18:00:16.582176+0000","last_clean":"2024-03-28T18:00:16.582176+0000","last_became_active":"2024-03-28T18:00:16.577824+0000","last_became_peered":"2024-03-28T18:00:16.577824+0000","last_unstale":"2024-03-28T18:00:16.582176+0000","last_undegraded":"2024-03-28T18:00:16.582176+0000","last_fullsized":"2024-03-28T18:00:16.582176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569631+0000","last_change":"2024-03-28T18:00:16.569631+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569631+0000","last_undegraded":"2024-03-28T18:00:16.569631+0000","last_fullsized":"2024-03-28T18:00:16.569631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588830+0000","last_change":"2024-03-28T18:00:16.584736+0000","last_active":"2024-03-28T18:00:16.588830+0000","last_peered":"2024-03-28T18:00:16.588830+0000","last_clean":"2024-03-28T18:00:16.588830+0000","last_became_active":"2024-03-28T18:00:16.584455+0000","last_became_peered":"2024-03-28T18:00:16.584455+0000","last_unstale":"2024-03-28T18:00:16.588830+0000","last_undegraded":"2024-03-28T18:00:16.588830+0000","last_fullsized":"2024-03-28T18:00:16.588830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557269+0000","last_change":"2024-03-28T18:00:16.557269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557269+0000","last_undegraded":"2024-03-28T18:00:16.557269+0000","last_fullsized":"2024-03-28T18:00:16.557269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.584256+0000","last_change":"2024-03-28T18:00:16.583053+0000","last_active":"2024-03-28T18:00:16.584256+0000","last_peered":"2024-03-28T18:00:16.584256+0000","last_clean":"2024-03-28T18:00:16.584256+0000","last_became_active":"2024-03-28T18:00:16.582775+0000","last_became_peered":"2024-03-28T18:00:16.582775+0000","last_unstale":"2024-03-28T18:00:16.584256+0000","last_undegraded":"2024-03-28T18:00:16.584256+0000","last_fullsized":"2024-03-28T18:00:16.584256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560345+0000","last_change":"2024-03-28T18:00:16.560345+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560345+0000","last_undegraded":"2024-03-28T18:00:16.560345+0000","last_fullsized":"2024-03-28T18:00:16.560345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579729+0000","last_change":"2024-03-28T18:00:16.575792+0000","last_active":"2024-03-28T18:00:16.579729+0000","last_peered":"2024-03-28T18:00:16.579729+0000","last_clean":"2024-03-28T18:00:16.579729+0000","last_became_active":"2024-03-28T18:00:16.575475+0000","last_became_peered":"2024-03-28T18:00:16.575475+0000","last_unstale":"2024-03-28T18:00:16.579729+0000","last_undegraded":"2024-03-28T18:00:16.579729+0000","last_fullsized":"2024-03-28T18:00:16.579729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562722+0000","last_change":"2024-03-28T18:00:16.562722+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562722+0000","last_undegraded":"2024-03-28T18:00:16.562722+0000","last_fullsized":"2024-03-28T18:00:16.562722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579125+0000","last_change":"2024-03-28T18:00:16.576446+0000","last_active":"2024-03-28T18:00:16.579125+0000","last_peered":"2024-03-28T18:00:16.579125+0000","last_clean":"2024-03-28T18:00:16.579125+0000","last_became_active":"2024-03-28T18:00:16.575444+0000","last_became_peered":"2024-03-28T18:00:16.575444+0000","last_unstale":"2024-03-28T18:00:16.579125+0000","last_undegraded":"2024-03-28T18:00:16.579125+0000","last_fullsized":"2024-03-28T18:00:16.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567784+0000","last_change":"2024-03-28T18:00:16.567784+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567784+0000","last_undegraded":"2024-03-28T18:00:16.567784+0000","last_fullsized":"2024-03-28T18:00:16.567784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.577769+0000","last_change":"2024-03-28T18:00:16.573496+0000","last_active":"2024-03-28T18:00:16.577769+0000","last_peered":"2024-03-28T18:00:16.577769+0000","last_clean":"2024-03-28T18:00:16.577769+0000","last_became_active":"2024-03-28T18:00:16.573041+0000","last_became_peered":"2024-03-28T18:00:16.573041+0000","last_unstale":"2024-03-28T18:00:16.577769+0000","last_undegraded":"2024-03-28T18:00:16.577769+0000","last_fullsized":"2024-03-28T18:00:16.577769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:56.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561746+0000","last_change":"2024-03-28T18:00:16.561746+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561746+0000","last_undegraded":"2024-03-28T18:00:16.561746+0000","last_fullsized":"2024-03-28T18:00:16.561746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583640+0000","last_change":"2024-03-28T18:00:16.581384+0000","last_active":"2024-03-28T18:00:16.583640+0000","last_peered":"2024-03-28T18:00:16.583640+0000","last_clean":"2024-03-28T18:00:16.583640+0000","last_became_active":"2024-03-28T18:00:16.581030+0000","last_became_peered":"2024-03-28T18:00:16.581030+0000","last_unstale":"2024-03-28T18:00:16.583640+0000","last_undegraded":"2024-03-28T18:00:16.583640+0000","last_fullsized":"2024-03-28T18:00:16.583640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569391+0000","last_change":"2024-03-28T18:00:16.569391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569391+0000","last_undegraded":"2024-03-28T18:00:16.569391+0000","last_fullsized":"2024-03-28T18:00:16.569391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.587915+0000","last_change":"2024-03-28T18:00:16.582503+0000","last_active":"2024-03-28T18:00:16.587915+0000","last_peered":"2024-03-28T18:00:16.587915+0000","last_clean":"2024-03-28T18:00:16.587915+0000","last_became_active":"2024-03-28T18:00:16.581354+0000","last_became_peered":"2024-03-28T18:00:16.581354+0000","last_unstale":"2024-03-28T18:00:16.587915+0000","last_undegraded":"2024-03-28T18:00:16.587915+0000","last_fullsized":"2024-03-28T18:00:16.587915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:02.785011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575167+0000","last_change":"2024-03-28T18:00:16.575167+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575167+0000","last_undegraded":"2024-03-28T18:00:16.575167+0000","last_fullsized":"2024-03-28T18:00:16.575167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592189+0000","last_change":"2024-03-28T18:00:16.587890+0000","last_active":"2024-03-28T18:00:16.592189+0000","last_peered":"2024-03-28T18:00:16.592189+0000","last_clean":"2024-03-28T18:00:16.592189+0000","last_became_active":"2024-03-28T18:00:16.587564+0000","last_became_peered":"2024-03-28T18:00:16.587564+0000","last_unstale":"2024-03-28T18:00:16.592189+0000","last_undegraded":"2024-03-28T18:00:16.592189+0000","last_fullsized":"2024-03-28T18:00:16.592189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576464+0000","last_change":"2024-03-28T18:00:16.576464+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576464+0000","last_undegraded":"2024-03-28T18:00:16.576464+0000","last_fullsized":"2024-03-28T18:00:16.576464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590990+0000","last_change":"2024-03-28T18:00:16.589804+0000","last_active":"2024-03-28T18:00:16.590990+0000","last_peered":"2024-03-28T18:00:16.590990+0000","last_clean":"2024-03-28T18:00:16.590990+0000","last_became_active":"2024-03-28T18:00:16.589477+0000","last_became_peered":"2024-03-28T18:00:16.589477+0000","last_unstale":"2024-03-28T18:00:16.590990+0000","last_undegraded":"2024-03-28T18:00:16.590990+0000","last_fullsized":"2024-03-28T18:00:16.590990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579800+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T18:00:16.579800+0000","last_peered":"2024-03-28T18:00:16.579800+0000","last_clean":"2024-03-28T18:00:16.579800+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T18:00:16.579800+0000","last_undegraded":"2024-03-28T18:00:16.579800+0000","last_fullsized":"2024-03-28T18:00:16.579800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:00:22.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:00:22.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.478+0000 7f130ea00640 1 Processor -- start 2024-03-28T18:00:22.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.479+0000 7f130ea00640 1 -- start start 2024-03-28T18:00:22.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.480+0000 7f130ea00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f13080cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.480+0000 7f130ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1308160110 con 0x7f13080cf3c0 2024-03-28T18:00:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.480+0000 7f130ea00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13081603c0 con 0x7f1308152f90 2024-03-28T18:00:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.480+0000 7f130ea00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13080cf830 con 0x7f130806ccb0 2024-03-28T18:00:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.481+0000 7f130ca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f13080cf3c0 0x7f13080b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60938/0 (socket says 172.21.15.62:60938) 2024-03-28T18:00:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.481+0000 7f130ca00640 1 -- 172.21.15.62:0/229056451 learned_addr learned my addr 172.21.15.62:0/229056451 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.481+0000 7f1307400640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f13080cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.481+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468025920 0 0) 0x7f1308160110 con 0x7f13080cf3c0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.481+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ec003520 con 0x7f13080cf3c0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597638199 0 0) 0x7f12ec003520 con 0x7f13080cf3c0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 msgr2=0x7f13080cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f13080cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 >> v1:172.21.15.81:6789/0 conn(0x7f1308152f90 legacy=0x7f130815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13081606a0 con 0x7f13080cf3c0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f1300001670 con 0x7f13080cf3c0 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 >> v1:172.21.15.62:6789/0 conn(0x7f13080cf3c0 legacy=0x7f13080b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f12ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec003fb0 0x7f12ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13080cf830 con 0x7f12ec004d40 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00bb00 con 0x7f12ec003fb0 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00bd00 con 0x7f130806ccb0 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f130ca00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1307e00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f12ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.482+0000 7f1307400640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec003fb0 0x7f12ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.483+0000 7f130ca00640 1 -- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 msgr2=0x7f12ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.483+0000 7f130ca00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130806ccb0 0x7f12ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.483+0000 7f130ca00640 1 -- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec003fb0 msgr2=0x7f12ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.483+0000 7f130ca00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec003fb0 0x7f12ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.483+0000 7f130ca00640 1 -- 172.21.15.62:0/229056451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec003520 con 0x7f12ec004d40 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.484+0000 7f130ca00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f130000c110 tx=0x7f12ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea7b2bfccf29ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.484+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300007c40 con 0x7f12ec004d40 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.484+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1300030070 con 0x7f12ec004d40 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.484+0000 7f1306a00640 1 -- 172.21.15.62:0/229056451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300005d70 con 0x7f12ec004d40 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 -- 172.21.15.62:0/229056451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12ec004d40 msgr2=0x7f12ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 --2- 172.21.15.62:0/229056451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12ec004d40 0x7f12ec0051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f130000c110 tx=0x7f12ec005830 comp rx=0 tx=0).stop 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ca00640 1 -- 172.21.15.62:0/229056451 reap_dead start 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 -- 172.21.15.62:0/229056451 shutdown_connections 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 -- 172.21.15.62:0/229056451 >> 172.21.15.62:0/229056451 conn(0x7f1308066fb0 msgr2=0x7f13080673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 -- 172.21.15.62:0/229056451 shutdown_connections 2024-03-28T18:00:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.485+0000 7f130ea00640 1 -- 172.21.15.62:0/229056451 wait complete. 2024-03-28T18:00:22.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.487+0000 7f130ea00640 1 Processor -- start 2024-03-28T18:00:22.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.487+0000 7f130ea00640 1 -- start start 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec004d40 0x7f1308146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130815ae50 0x7f130815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130815c5d0 con 0x7f1308071e10 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13080b3f90 con 0x7f12ec004d40 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f130ea00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130815c750 con 0x7f130815ae50 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42680/0 (socket says 172.21.15.62:42680) 2024-03-28T18:00:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307400640 1 -- 172.21.15.62:0/1365448823 learned_addr learned my addr 172.21.15.62:0/1365448823 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307e00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec004d40 0x7f1308146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307400640 1 -- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130815ae50 msgr2=0x7f130815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.488+0000 7f1307400640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130815ae50 0x7f130815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f1307400640 1 -- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec004d40 msgr2=0x7f1308146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f1307400640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec004d40 0x7f1308146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f1307400640 1 -- 172.21.15.62:0/1365448823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308070d70 con 0x7f1308071e10 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f1307400640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12f0011530 tx=0x7f12f00119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0015870 con 0x7f1308071e10 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0015a10 con 0x7f1308071e10 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0015dc0 con 0x7f1308071e10 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1308071050 con 0x7f1308071e10 2024-03-28T18:00:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.489+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1308071610 con 0x7f1308071e10 2024-03-28T18:00:22.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.491+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13080680c0 con 0x7f1308071e10 2024-03-28T18:00:22.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.492+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f12f0013400 con 0x7f1308071e10 2024-03-28T18:00:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.492+0000 7f12ffe00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 0x7f12e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.493+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12f008b3c0 con 0x7f1308071e10 2024-03-28T18:00:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.493+0000 7f1307e00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 0x7f12e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.494+0000 7f1307e00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 0x7f12e406e090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12f8003fd0 tx=0x7f12f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:22.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.502+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f12f0019030 con 0x7f1308071e10 2024-03-28T18:00:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.701+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13080adfb0 con 0x7f12e406bbd0 2024-03-28T18:00:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.719+0000 7f12ffe00640 1 -- 172.21.15.62:0/1365448823 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f13080adfb0 con 0x7f12e406bbd0 2024-03-28T18:00:22.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:22.723 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 msgr2=0x7f12e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 0x7f12e406e090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12f8003fd0 tx=0x7f12f8009040 comp rx=0 tx=0).stop 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 msgr2=0x7f13080724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12f0011530 tx=0x7f12f00119f0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 shutdown_connections 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f12e406bbd0 0x7f12e406e090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f130815ae50 0x7f130815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308071e10 0x7f13080724d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 --2- 172.21.15.62:0/1365448823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12ec004d40 0x7f1308146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:22.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 >> 172.21.15.62:0/1365448823 conn(0x7f1308066fb0 msgr2=0x7f13080b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:22.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 shutdown_connections 2024-03-28T18:00:22.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:22.725+0000 7f130ea00640 1 -- 172.21.15.62:0/1365448823 wait complete. 2024-03-28T18:00:22.743 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-28T18:00:22.335925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3960832,"data_stored":2242797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001793"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569583+0000","last_change":"2024-03-28T18:00:16.569583+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569583+0000","last_undegraded":"2024-03-28T18:00:16.569583+0000","last_fullsized":"2024-03-28T18:00:16.569583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579654+0000","last_change":"2024-03-28T18:00:16.579654+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579654+0000","last_undegraded":"2024-03-28T18:00:16.579654+0000","last_fullsized":"2024-03-28T18:00:16.579654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561545+0000","last_change":"2024-03-28T18:00:16.561545+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561545+0000","last_undegraded":"2024-03-28T18:00:16.561545+0000","last_fullsized":"2024-03-28T18:00:16.561545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571390+0000","last_change":"2024-03-28T18:00:16.571390+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571390+0000","last_undegraded":"2024-03-28T18:00:16.571390+0000","last_fullsized":"2024-03-28T18:00:16.571390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557319+0000","last_change":"2024-03-28T18:00:16.557319+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557319+0000","last_undegraded":"2024-03-28T18:00:16.557319+0000","last_fullsized":"2024-03-28T18:00:16.557319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.580779+0000","last_change":"2024-03-28T18:00:16.580779+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.580779+0000","last_undegraded":"2024-03-28T18:00:16.580779+0000","last_fullsized":"2024-03-28T18:00:16.580779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568442+0000","last_change":"2024-03-28T18:00:16.568442+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568442+0000","last_undegraded":"2024-03-28T18:00:16.568442+0000","last_fullsized":"2024-03-28T18:00:16.568442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575897+0000","last_change":"2024-03-28T18:00:16.575897+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575897+0000","last_undegraded":"2024-03-28T18:00:16.575897+0000","last_fullsized":"2024-03-28T18:00:16.575897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576562+0000","last_change":"2024-03-28T18:00:16.576562+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576562+0000","last_undegraded":"2024-03-28T18:00:16.576562+0000","last_fullsized":"2024-03-28T18:00:16.576562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568412+0000","last_change":"2024-03-28T18:00:16.568412+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568412+0000","last_undegraded":"2024-03-28T18:00:16.568412+0000","last_fullsized":"2024-03-28T18:00:16.568412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569231+0000","last_change":"2024-03-28T18:00:16.569231+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569231+0000","last_undegraded":"2024-03-28T18:00:16.569231+0000","last_fullsized":"2024-03-28T18:00:16.569231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559639+0000","last_change":"2024-03-28T18:00:16.559639+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559639+0000","last_undegraded":"2024-03-28T18:00:16.559639+0000","last_fullsized":"2024-03-28T18:00:16.559639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556645+0000","last_change":"2024-03-28T18:00:16.556645+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556645+0000","last_undegraded":"2024-03-28T18:00:16.556645+0000","last_fullsized":"2024-03-28T18:00:16.556645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572147+0000","last_change":"2024-03-28T18:00:16.572147+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572147+0000","last_undegraded":"2024-03-28T18:00:16.572147+0000","last_fullsized":"2024-03-28T18:00:16.572147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557955+0000","last_change":"2024-03-28T18:00:16.557955+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557955+0000","last_undegraded":"2024-03-28T18:00:16.557955+0000","last_fullsized":"2024-03-28T18:00:16.557955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567619+0000","last_change":"2024-03-28T18:00:16.567619+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567619+0000","last_undegraded":"2024-03-28T18:00:16.567619+0000","last_fullsized":"2024-03-28T18:00:16.567619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567977+0000","last_change":"2024-03-28T18:00:16.567977+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567977+0000","last_undegraded":"2024-03-28T18:00:16.567977+0000","last_fullsized":"2024-03-28T18:00:16.567977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.577391+0000","last_change":"2024-03-28T18:00:16.577391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.577391+0000","last_undegraded":"2024-03-28T18:00:16.577391+0000","last_fullsized":"2024-03-28T18:00:16.577391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573104+0000","last_change":"2024-03-28T18:00:16.573104+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573104+0000","last_undegraded":"2024-03-28T18:00:16.573104+0000","last_fullsized":"2024-03-28T18:00:16.573104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555854+0000","last_change":"2024-03-28T18:00:16.555854+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555854+0000","last_undegraded":"2024-03-28T18:00:16.555854+0000","last_fullsized":"2024-03-28T18:00:16.555854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576380+0000","last_change":"2024-03-28T18:00:16.576380+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576380+0000","last_undegraded":"2024-03-28T18:00:16.576380+0000","last_fullsized":"2024-03-28T18:00:16.576380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571362+0000","last_change":"2024-03-28T18:00:16.571362+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571362+0000","last_undegraded":"2024-03-28T18:00:16.571362+0000","last_fullsized":"2024-03-28T18:00:16.571362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561762+0000","last_change":"2024-03-28T18:00:16.561762+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561762+0000","last_undegraded":"2024-03-28T18:00:16.561762+0000","last_fullsized":"2024-03-28T18:00:16.561762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558028+0000","last_change":"2024-03-28T18:00:16.558028+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558028+0000","last_undegraded":"2024-03-28T18:00:16.558028+0000","last_fullsized":"2024-03-28T18:00:16.558028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569204+0000","last_change":"2024-03-28T18:00:16.569204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569204+0000","last_undegraded":"2024-03-28T18:00:16.569204+0000","last_fullsized":"2024-03-28T18:00:16.569204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576919+0000","last_change":"2024-03-28T18:00:16.576919+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576919+0000","last_undegraded":"2024-03-28T18:00:16.576919+0000","last_fullsized":"2024-03-28T18:00:16.576919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566641+0000","last_change":"2024-03-28T18:00:16.566641+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566641+0000","last_undegraded":"2024-03-28T18:00:16.566641+0000","last_fullsized":"2024-03-28T18:00:16.566641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558269+0000","last_change":"2024-03-28T18:00:16.558269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558269+0000","last_undegraded":"2024-03-28T18:00:16.558269+0000","last_fullsized":"2024-03-28T18:00:16.558269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575581+0000","last_change":"2024-03-28T18:00:16.575581+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575581+0000","last_undegraded":"2024-03-28T18:00:16.575581+0000","last_fullsized":"2024-03-28T18:00:16.575581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572204+0000","last_change":"2024-03-28T18:00:16.572204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572204+0000","last_undegraded":"2024-03-28T18:00:16.572204+0000","last_fullsized":"2024-03-28T18:00:16.572204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571667+0000","last_change":"2024-03-28T18:00:16.571667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571667+0000","last_undegraded":"2024-03-28T18:00:16.571667+0000","last_fullsized":"2024-03-28T18:00:16.571667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574076+0000","last_change":"2024-03-28T18:00:16.574076+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574076+0000","last_undegraded":"2024-03-28T18:00:16.574076+0000","last_fullsized":"2024-03-28T18:00:16.574076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562017+0000","last_change":"2024-03-28T18:00:16.562017+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562017+0000","last_undegraded":"2024-03-28T18:00:16.562017+0000","last_fullsized":"2024-03-28T18:00:16.562017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588360+0000","last_change":"2024-03-28T18:00:16.586041+0000","last_active":"2024-03-28T18:00:16.588360+0000","last_peered":"2024-03-28T18:00:16.588360+0000","last_clean":"2024-03-28T18:00:16.588360+0000","last_became_active":"2024-03-28T18:00:16.585654+0000","last_became_peered":"2024-03-28T18:00:16.585654+0000","last_unstale":"2024-03-28T18:00:16.588360+0000","last_undegraded":"2024-03-28T18:00:16.588360+0000","last_fullsized":"2024-03-28T18:00:16.588360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567054+0000","last_change":"2024-03-28T18:00:16.567054+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567054+0000","last_undegraded":"2024-03-28T18:00:16.567054+0000","last_fullsized":"2024-03-28T18:00:16.567054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586926+0000","last_change":"2024-03-28T18:00:16.582651+0000","last_active":"2024-03-28T18:00:16.586926+0000","last_peered":"2024-03-28T18:00:16.586926+0000","last_clean":"2024-03-28T18:00:16.586926+0000","last_became_active":"2024-03-28T18:00:16.582248+0000","last_became_peered":"2024-03-28T18:00:16.582248+0000","last_unstale":"2024-03-28T18:00:16.586926+0000","last_undegraded":"2024-03-28T18:00:16.586926+0000","last_fullsized":"2024-03-28T18:00:16.586926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571368+0000","last_change":"2024-03-28T18:00:16.571368+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571368+0000","last_undegraded":"2024-03-28T18:00:16.571368+0000","last_fullsized":"2024-03-28T18:00:16.571368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592321+0000","last_change":"2024-03-28T18:00:16.588264+0000","last_active":"2024-03-28T18:00:16.592321+0000","last_peered":"2024-03-28T18:00:16.592321+0000","last_clean":"2024-03-28T18:00:16.592321+0000","last_became_active":"2024-03-28T18:00:16.587925+0000","last_became_peered":"2024-03-28T18:00:16.587925+0000","last_unstale":"2024-03-28T18:00:16.592321+0000","last_undegraded":"2024-03-28T18:00:16.592321+0000","last_fullsized":"2024-03-28T18:00:16.592321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565721+0000","last_change":"2024-03-28T18:00:16.565721+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565721+0000","last_undegraded":"2024-03-28T18:00:16.565721+0000","last_fullsized":"2024-03-28T18:00:16.565721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579273+0000","last_change":"2024-03-28T18:00:16.577746+0000","last_active":"2024-03-28T18:00:16.579273+0000","last_peered":"2024-03-28T18:00:16.579273+0000","last_clean":"2024-03-28T18:00:16.579273+0000","last_became_active":"2024-03-28T18:00:16.577340+0000","last_became_peered":"2024-03-28T18:00:16.577340+0000","last_unstale":"2024-03-28T18:00:16.579273+0000","last_undegraded":"2024-03-28T18:00:16.579273+0000","last_fullsized":"2024-03-28T18:00:16.579273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570787+0000","last_change":"2024-03-28T18:00:16.570787+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570787+0000","last_undegraded":"2024-03-28T18:00:16.570787+0000","last_fullsized":"2024-03-28T18:00:16.570787+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593331+0000","last_change":"2024-03-28T18:00:16.591523+0000","last_active":"2024-03-28T18:00:16.593331+0000","last_peered":"2024-03-28T18:00:16.593331+0000","last_clean":"2024-03-28T18:00:16.593331+0000","last_became_active":"2024-03-28T18:00:16.591203+0000","last_became_peered":"2024-03-28T18:00:16.591203+0000","last_unstale":"2024-03-28T18:00:16.593331+0000","last_undegraded":"2024-03-28T18:00:16.593331+0000","last_fullsized":"2024-03-28T18:00:16.593331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:13.152406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559660+0000","last_change":"2024-03-28T18:00:16.559660+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559660+0000","last_undegraded":"2024-03-28T18:00:16.559660+0000","last_fullsized":"2024-03-28T18:00:16.559660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593355+0000","last_change":"2024-03-28T18:00:16.591608+0000","last_active":"2024-03-28T18:00:16.593355+0000","last_peered":"2024-03-28T18:00:16.593355+0000","last_clean":"2024-03-28T18:00:16.593355+0000","last_became_active":"2024-03-28T18:00:16.591289+0000","last_became_peered":"2024-03-28T18:00:16.591289+0000","last_unstale":"2024-03-28T18:00:16.593355+0000","last_undegraded":"2024-03-28T18:00:16.593355+0000","last_fullsized":"2024-03-28T18:00:16.593355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:23.345799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574763+0000","last_change":"2024-03-28T18:00:16.574763+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574763+0000","last_undegraded":"2024-03-28T18:00:16.574763+0000","last_fullsized":"2024-03-28T18:00:16.574763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589450+0000","last_change":"2024-03-28T18:00:16.581766+0000","last_active":"2024-03-28T18:00:16.589450+0000","last_peered":"2024-03-28T18:00:16.589450+0000","last_clean":"2024-03-28T18:00:16.589450+0000","last_became_active":"2024-03-28T18:00:16.581506+0000","last_became_peered":"2024-03-28T18:00:16.581506+0000","last_unstale":"2024-03-28T18:00:16.589450+0000","last_undegraded":"2024-03-28T18:00:16.589450+0000","last_fullsized":"2024-03-28T18:00:16.589450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.564269+0000","last_change":"2024-03-28T18:00:16.564269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.564269+0000","last_undegraded":"2024-03-28T18:00:16.564269+0000","last_fullsized":"2024-03-28T18:00:16.564269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590746+0000","last_change":"2024-03-28T18:00:16.589309+0000","last_active":"2024-03-28T18:00:16.590746+0000","last_peered":"2024-03-28T18:00:16.590746+0000","last_clean":"2024-03-28T18:00:16.590746+0000","last_became_active":"2024-03-28T18:00:16.588995+0000","last_became_peered":"2024-03-28T18:00:16.588995+0000","last_unstale":"2024-03-28T18:00:16.590746+0000","last_undegraded":"2024-03-28T18:00:16.590746+0000","last_fullsized":"2024-03-28T18:00:16.590746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:10.952085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560002+0000","last_change":"2024-03-28T18:00:16.560002+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560002+0000","last_undegraded":"2024-03-28T18:00:16.560002+0000","last_fullsized":"2024-03-28T18:00:16.560002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592334+0000","last_change":"2024-03-28T18:00:16.588159+0000","last_active":"2024-03-28T18:00:16.592334+0000","last_peered":"2024-03-28T18:00:16.592334+0000","last_clean":"2024-03-28T18:00:16.592334+0000","last_became_active":"2024-03-28T18:00:16.587891+0000","last_became_peered":"2024-03-28T18:00:16.587891+0000","last_unstale":"2024-03-28T18:00:16.592334+0000","last_undegraded":"2024-03-28T18:00:16.592334+0000","last_fullsized":"2024-03-28T18:00:16.592334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574440+0000","last_change":"2024-03-28T18:00:16.574440+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574440+0000","last_undegraded":"2024-03-28T18:00:16.574440+0000","last_fullsized":"2024-03-28T18:00:16.574440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592843+0000","last_change":"2024-03-28T18:00:16.590209+0000","last_active":"2024-03-28T18:00:16.592843+0000","last_peered":"2024-03-28T18:00:16.592843+0000","last_clean":"2024-03-28T18:00:16.592843+0000","last_became_active":"2024-03-28T18:00:16.589874+0000","last_became_peered":"2024-03-28T18:00:16.589874+0000","last_unstale":"2024-03-28T18:00:16.592843+0000","last_undegraded":"2024-03-28T18:00:16.592843+0000","last_fullsized":"2024-03-28T18:00:16.592843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573906+0000","last_change":"2024-03-28T18:00:16.573906+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573906+0000","last_undegraded":"2024-03-28T18:00:16.573906+0000","last_fullsized":"2024-03-28T18:00:16.573906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588152+0000","last_change":"2024-03-28T18:00:16.583925+0000","last_active":"2024-03-28T18:00:16.588152+0000","last_peered":"2024-03-28T18:00:16.588152+0000","last_clean":"2024-03-28T18:00:16.588152+0000","last_became_active":"2024-03-28T18:00:16.583635+0000","last_became_peered":"2024-03-28T18:00:16.583635+0000","last_unstale":"2024-03-28T18:00:16.588152+0000","last_undegraded":"2024-03-28T18:00:16.588152+0000","last_fullsized":"2024-03-28T18:00:16.588152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562694+0000","last_change":"2024-03-28T18:00:16.562694+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562694+0000","last_undegraded":"2024-03-28T18:00:16.562694+0000","last_fullsized":"2024-03-28T18:00:16.562694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.597940+0000","last_change":"2024-03-28T18:00:16.593104+0000","last_active":"2024-03-28T18:00:16.597940+0000","last_peered":"2024-03-28T18:00:16.597940+0000","last_clean":"2024-03-28T18:00:16.597940+0000","last_became_active":"2024-03-28T18:00:16.592771+0000","last_became_peered":"2024-03-28T18:00:16.592771+0000","last_unstale":"2024-03-28T18:00:16.597940+0000","last_undegraded":"2024-03-28T18:00:16.597940+0000","last_fullsized":"2024-03-28T18:00:16.597940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565690+0000","last_change":"2024-03-28T18:00:16.565690+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565690+0000","last_undegraded":"2024-03-28T18:00:16.565690+0000","last_fullsized":"2024-03-28T18:00:16.565690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589009+0000","last_change":"2024-03-28T18:00:16.584849+0000","last_active":"2024-03-28T18:00:16.589009+0000","last_peered":"2024-03-28T18:00:16.589009+0000","last_clean":"2024-03-28T18:00:16.589009+0000","last_became_active":"2024-03-28T18:00:16.584543+0000","last_became_peered":"2024-03-28T18:00:16.584543+0000","last_unstale":"2024-03-28T18:00:16.589009+0000","last_undegraded":"2024-03-28T18:00:16.589009+0000","last_fullsized":"2024-03-28T18:00:16.589009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:28:00.623248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579339+0000","last_change":"2024-03-28T18:00:16.579339+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579339+0000","last_undegraded":"2024-03-28T18:00:16.579339+0000","last_fullsized":"2024-03-28T18:00:16.579339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583382+0000","last_change":"2024-03-28T18:00:16.580448+0000","last_active":"2024-03-28T18:00:16.583382+0000","last_peered":"2024-03-28T18:00:16.583382+0000","last_clean":"2024-03-28T18:00:16.583382+0000","last_became_active":"2024-03-28T18:00:16.579745+0000","last_became_peered":"2024-03-28T18:00:16.579745+0000","last_unstale":"2024-03-28T18:00:16.583382+0000","last_undegraded":"2024-03-28T18:00:16.583382+0000","last_fullsized":"2024-03-28T18:00:16.583382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571904+0000","last_change":"2024-03-28T18:00:16.571904+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571904+0000","last_undegraded":"2024-03-28T18:00:16.571904+0000","last_fullsized":"2024-03-28T18:00:16.571904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592117+0000","last_change":"2024-03-28T18:00:16.587497+0000","last_active":"2024-03-28T18:00:16.592117+0000","last_peered":"2024-03-28T18:00:16.592117+0000","last_clean":"2024-03-28T18:00:16.592117+0000","last_became_active":"2024-03-28T18:00:16.586180+0000","last_became_peered":"2024-03-28T18:00:16.586180+0000","last_unstale":"2024-03-28T18:00:16.592117+0000","last_undegraded":"2024-03-28T18:00:16.592117+0000","last_fullsized":"2024-03-28T18:00:16.592117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566667+0000","last_change":"2024-03-28T18:00:16.566667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566667+0000","last_undegraded":"2024-03-28T18:00:16.566667+0000","last_fullsized":"2024-03-28T18:00:16.566667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589771+0000","last_change":"2024-03-28T18:00:16.581297+0000","last_active":"2024-03-28T18:00:16.589771+0000","last_peered":"2024-03-28T18:00:16.589771+0000","last_clean":"2024-03-28T18:00:16.589771+0000","last_became_active":"2024-03-28T18:00:16.580937+0000","last_became_peered":"2024-03-28T18:00:16.580937+0000","last_unstale":"2024-03-28T18:00:16.589771+0000","last_undegraded":"2024-03-28T18:00:16.589771+0000","last_fullsized":"2024-03-28T18:00:16.589771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571676+0000","last_change":"2024-03-28T18:00:16.571676+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571676+0000","last_undegraded":"2024-03-28T18:00:16.571676+0000","last_fullsized":"2024-03-28T18:00:16.571676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592286+0000","last_change":"2024-03-28T18:00:16.589262+0000","last_active":"2024-03-28T18:00:16.592286+0000","last_peered":"2024-03-28T18:00:16.592286+0000","last_clean":"2024-03-28T18:00:16.592286+0000","last_became_active":"2024-03-28T18:00:16.588786+0000","last_became_peered":"2024-03-28T18:00:16.588786+0000","last_unstale":"2024-03-28T18:00:16.592286+0000","last_undegraded":"2024-03-28T18:00:16.592286+0000","last_fullsized":"2024-03-28T18:00:16.592286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561624+0000","last_change":"2024-03-28T18:00:16.561624+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561624+0000","last_undegraded":"2024-03-28T18:00:16.561624+0000","last_fullsized":"2024-03-28T18:00:16.561624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589122+0000","last_change":"2024-03-28T18:00:16.587380+0000","last_active":"2024-03-28T18:00:16.589122+0000","last_peered":"2024-03-28T18:00:16.589122+0000","last_clean":"2024-03-28T18:00:16.589122+0000","last_became_active":"2024-03-28T18:00:16.586842+0000","last_became_peered":"2024-03-28T18:00:16.586842+0000","last_unstale":"2024-03-28T18:00:16.589122+0000","last_undegraded":"2024-03-28T18:00:16.589122+0000","last_fullsized":"2024-03-28T18:00:16.589122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.737137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570537+0000","last_change":"2024-03-28T18:00:16.570537+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570537+0000","last_undegraded":"2024-03-28T18:00:16.570537+0000","last_fullsized":"2024-03-28T18:00:16.570537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588036+0000","last_change":"2024-03-28T18:00:16.581961+0000","last_active":"2024-03-28T18:00:16.588036+0000","last_peered":"2024-03-28T18:00:16.588036+0000","last_clean":"2024-03-28T18:00:16.588036+0000","last_became_active":"2024-03-28T18:00:16.581694+0000","last_became_peered":"2024-03-28T18:00:16.581694+0000","last_unstale":"2024-03-28T18:00:16.588036+0000","last_undegraded":"2024-03-28T18:00:16.588036+0000","last_fullsized":"2024-03-28T18:00:16.588036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:58:38.567836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556811+0000","last_change":"2024-03-28T18:00:16.556811+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556811+0000","last_undegraded":"2024-03-28T18:00:16.556811+0000","last_fullsized":"2024-03-28T18:00:16.556811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.591372+0000","last_change":"2024-03-28T18:00:16.590452+0000","last_active":"2024-03-28T18:00:16.591372+0000","last_peered":"2024-03-28T18:00:16.591372+0000","last_clean":"2024-03-28T18:00:16.591372+0000","last_became_active":"2024-03-28T18:00:16.590148+0000","last_became_peered":"2024-03-28T18:00:16.590148+0000","last_unstale":"2024-03-28T18:00:16.591372+0000","last_undegraded":"2024-03-28T18:00:16.591372+0000","last_fullsized":"2024-03-28T18:00:16.591372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:59.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555930+0000","last_change":"2024-03-28T18:00:16.555930+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555930+0000","last_undegraded":"2024-03-28T18:00:16.555930+0000","last_fullsized":"2024-03-28T18:00:16.555930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.578677+0000","last_change":"2024-03-28T18:00:16.574301+0000","last_active":"2024-03-28T18:00:16.578677+0000","last_peered":"2024-03-28T18:00:16.578677+0000","last_clean":"2024-03-28T18:00:16.578677+0000","last_became_active":"2024-03-28T18:00:16.573671+0000","last_became_peered":"2024-03-28T18:00:16.573671+0000","last_unstale":"2024-03-28T18:00:16.578677+0000","last_undegraded":"2024-03-28T18:00:16.578677+0000","last_fullsized":"2024-03-28T18:00:16.578677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555199+0000","last_change":"2024-03-28T18:00:16.555199+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555199+0000","last_undegraded":"2024-03-28T18:00:16.555199+0000","last_fullsized":"2024-03-28T18:00:16.555199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586129+0000","last_change":"2024-03-28T18:00:16.584421+0000","last_active":"2024-03-28T18:00:16.586129+0000","last_peered":"2024-03-28T18:00:16.586129+0000","last_clean":"2024-03-28T18:00:16.586129+0000","last_became_active":"2024-03-28T18:00:16.584113+0000","last_became_peered":"2024-03-28T18:00:16.584113+0000","last_unstale":"2024-03-28T18:00:16.586129+0000","last_undegraded":"2024-03-28T18:00:16.586129+0000","last_fullsized":"2024-03-28T18:00:16.586129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556334+0000","last_change":"2024-03-28T18:00:16.556334+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556334+0000","last_undegraded":"2024-03-28T18:00:16.556334+0000","last_fullsized":"2024-03-28T18:00:16.556334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.582176+0000","last_change":"2024-03-28T18:00:16.578260+0000","last_active":"2024-03-28T18:00:16.582176+0000","last_peered":"2024-03-28T18:00:16.582176+0000","last_clean":"2024-03-28T18:00:16.582176+0000","last_became_active":"2024-03-28T18:00:16.577824+0000","last_became_peered":"2024-03-28T18:00:16.577824+0000","last_unstale":"2024-03-28T18:00:16.582176+0000","last_undegraded":"2024-03-28T18:00:16.582176+0000","last_fullsized":"2024-03-28T18:00:16.582176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569631+0000","last_change":"2024-03-28T18:00:16.569631+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569631+0000","last_undegraded":"2024-03-28T18:00:16.569631+0000","last_fullsized":"2024-03-28T18:00:16.569631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588830+0000","last_change":"2024-03-28T18:00:16.584736+0000","last_active":"2024-03-28T18:00:16.588830+0000","last_peered":"2024-03-28T18:00:16.588830+0000","last_clean":"2024-03-28T18:00:16.588830+0000","last_became_active":"2024-03-28T18:00:16.584455+0000","last_became_peered":"2024-03-28T18:00:16.584455+0000","last_unstale":"2024-03-28T18:00:16.588830+0000","last_undegraded":"2024-03-28T18:00:16.588830+0000","last_fullsized":"2024-03-28T18:00:16.588830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557269+0000","last_change":"2024-03-28T18:00:16.557269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557269+0000","last_undegraded":"2024-03-28T18:00:16.557269+0000","last_fullsized":"2024-03-28T18:00:16.557269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.584256+0000","last_change":"2024-03-28T18:00:16.583053+0000","last_active":"2024-03-28T18:00:16.584256+0000","last_peered":"2024-03-28T18:00:16.584256+0000","last_clean":"2024-03-28T18:00:16.584256+0000","last_became_active":"2024-03-28T18:00:16.582775+0000","last_became_peered":"2024-03-28T18:00:16.582775+0000","last_unstale":"2024-03-28T18:00:16.584256+0000","last_undegraded":"2024-03-28T18:00:16.584256+0000","last_fullsized":"2024-03-28T18:00:16.584256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560345+0000","last_change":"2024-03-28T18:00:16.560345+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560345+0000","last_undegraded":"2024-03-28T18:00:16.560345+0000","last_fullsized":"2024-03-28T18:00:16.560345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579729+0000","last_change":"2024-03-28T18:00:16.575792+0000","last_active":"2024-03-28T18:00:16.579729+0000","last_peered":"2024-03-28T18:00:16.579729+0000","last_clean":"2024-03-28T18:00:16.579729+0000","last_became_active":"2024-03-28T18:00:16.575475+0000","last_became_peered":"2024-03-28T18:00:16.575475+0000","last_unstale":"2024-03-28T18:00:16.579729+0000","last_undegraded":"2024-03-28T18:00:16.579729+0000","last_fullsized":"2024-03-28T18:00:16.579729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562722+0000","last_change":"2024-03-28T18:00:16.562722+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562722+0000","last_undegraded":"2024-03-28T18:00:16.562722+0000","last_fullsized":"2024-03-28T18:00:16.562722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579125+0000","last_change":"2024-03-28T18:00:16.576446+0000","last_active":"2024-03-28T18:00:16.579125+0000","last_peered":"2024-03-28T18:00:16.579125+0000","last_clean":"2024-03-28T18:00:16.579125+0000","last_became_active":"2024-03-28T18:00:16.575444+0000","last_became_peered":"2024-03-28T18:00:16.575444+0000","last_unstale":"2024-03-28T18:00:16.579125+0000","last_undegraded":"2024-03-28T18:00:16.579125+0000","last_fullsized":"2024-03-28T18:00:16.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567784+0000","last_change":"2024-03-28T18:00:16.567784+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567784+0000","last_undegraded":"2024-03-28T18:00:16.567784+0000","last_fullsized":"2024-03-28T18:00:16.567784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.577769+0000","last_change":"2024-03-28T18:00:16.573496+0000","last_active":"2024-03-28T18:00:16.577769+0000","last_peered":"2024-03-28T18:00:16.577769+0000","last_clean":"2024-03-28T18:00:16.577769+0000","last_became_active":"2024-03-28T18:00:16.573041+0000","last_became_peered":"2024-03-28T18:00:16.573041+0000","last_unstale":"2024-03-28T18:00:16.577769+0000","last_undegraded":"2024-03-28T18:00:16.577769+0000","last_fullsized":"2024-03-28T18:00:16.577769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:56.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561746+0000","last_change":"2024-03-28T18:00:16.561746+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561746+0000","last_undegraded":"2024-03-28T18:00:16.561746+0000","last_fullsized":"2024-03-28T18:00:16.561746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583640+0000","last_change":"2024-03-28T18:00:16.581384+0000","last_active":"2024-03-28T18:00:16.583640+0000","last_peered":"2024-03-28T18:00:16.583640+0000","last_clean":"2024-03-28T18:00:16.583640+0000","last_became_active":"2024-03-28T18:00:16.581030+0000","last_became_peered":"2024-03-28T18:00:16.581030+0000","last_unstale":"2024-03-28T18:00:16.583640+0000","last_undegraded":"2024-03-28T18:00:16.583640+0000","last_fullsized":"2024-03-28T18:00:16.583640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569391+0000","last_change":"2024-03-28T18:00:16.569391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569391+0000","last_undegraded":"2024-03-28T18:00:16.569391+0000","last_fullsized":"2024-03-28T18:00:16.569391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.587915+0000","last_change":"2024-03-28T18:00:16.582503+0000","last_active":"2024-03-28T18:00:16.587915+0000","last_peered":"2024-03-28T18:00:16.587915+0000","last_clean":"2024-03-28T18:00:16.587915+0000","last_became_active":"2024-03-28T18:00:16.581354+0000","last_became_peered":"2024-03-28T18:00:16.581354+0000","last_unstale":"2024-03-28T18:00:16.587915+0000","last_undegraded":"2024-03-28T18:00:16.587915+0000","last_fullsized":"2024-03-28T18:00:16.587915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:02.785011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575167+0000","last_change":"2024-03-28T18:00:16.575167+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575167+0000","last_undegraded":"2024-03-28T18:00:16.575167+0000","last_fullsized":"2024-03-28T18:00:16.575167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592189+0000","last_change":"2024-03-28T18:00:16.587890+0000","last_active":"2024-03-28T18:00:16.592189+0000","last_peered":"2024-03-28T18:00:16.592189+0000","last_clean":"2024-03-28T18:00:16.592189+0000","last_became_active":"2024-03-28T18:00:16.587564+0000","last_became_peered":"2024-03-28T18:00:16.587564+0000","last_unstale":"2024-03-28T18:00:16.592189+0000","last_undegraded":"2024-03-28T18:00:16.592189+0000","last_fullsized":"2024-03-28T18:00:16.592189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576464+0000","last_change":"2024-03-28T18:00:16.576464+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576464+0000","last_undegraded":"2024-03-28T18:00:16.576464+0000","last_fullsized":"2024-03-28T18:00:16.576464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590990+0000","last_change":"2024-03-28T18:00:16.589804+0000","last_active":"2024-03-28T18:00:16.590990+0000","last_peered":"2024-03-28T18:00:16.590990+0000","last_clean":"2024-03-28T18:00:16.590990+0000","last_became_active":"2024-03-28T18:00:16.589477+0000","last_became_peered":"2024-03-28T18:00:16.589477+0000","last_unstale":"2024-03-28T18:00:16.590990+0000","last_undegraded":"2024-03-28T18:00:16.590990+0000","last_fullsized":"2024-03-28T18:00:16.590990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579800+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T18:00:16.579800+0000","last_peered":"2024-03-28T18:00:16.579800+0000","last_clean":"2024-03-28T18:00:16.579800+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T18:00:16.579800+0000","last_undegraded":"2024-03-28T18:00:16.579800+0000","last_fullsized":"2024-03-28T18:00:16.579800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:00:22.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:00:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.004+0000 7fe0ecc00640 1 Processor -- start 2024-03-28T18:00:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.004+0000 7fe0ecc00640 1 -- start start 2024-03-28T18:00:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.006+0000 7fe0ecc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0e8154fa0 0x7fe0e815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.006+0000 7fe0ecc00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 0x7fe0e80aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.006+0000 7fe0ecc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e80ae3e0 con 0x7fe0e80b4240 2024-03-28T18:00:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.006+0000 7fe0ecc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e80b5250 con 0x7fe0e8154fa0 2024-03-28T18:00:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.006+0000 7fe0ecc00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e80b49b0 con 0x7fe0e8071c30 2024-03-28T18:00:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0e8154fa0 0x7fe0e815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe0e80b4240 0x7fe0e8071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60952/0 (socket says 172.21.15.62:60952) 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5400640 1 -- 172.21.15.62:0/3327395864 learned_addr learned my addr 172.21.15.62:0/3327395864 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e6800640 1 --2- 172.21.15.62:0/3327395864 >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 0x7fe0e80aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5e00640 1 -- 172.21.15.62:0/3327395864 >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 msgr2=0x7fe0e80aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5e00640 1 --2- 172.21.15.62:0/3327395864 >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 0x7fe0e80aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5e00640 1 -- 172.21.15.62:0/3327395864 >> v1:172.21.15.62:6789/0 conn(0x7fe0e80b4240 legacy=0x7fe0e8071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e5e00640 1 -- 172.21.15.62:0/3327395864 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80b4c90 con 0x7fe0e8154fa0 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e4a00640 1 -- 172.21.15.62:0/3327395864 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051540817 0 0) 0x7fe0d4001110 con 0x7fe0e80b4240 2024-03-28T18:00:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.007+0000 7fe0e6800640 1 --2- 172.21.15.62:0/3327395864 >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 0x7fe0e80aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.008+0000 7fe0e5e00640 1 --2- 172.21.15.62:0/3327395864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815e650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80cfeb0 tx=0x7fe0d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709c5ea511a38673 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0e4a00640 1 -- 172.21.15.62:0/3327395864 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80315a0 con 0x7fe0e8154fa0 2024-03-28T18:00:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0e4a00640 1 -- 172.21.15.62:0/3327395864 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d8037070 con 0x7fe0e8154fa0 2024-03-28T18:00:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0e4a00640 1 -- 172.21.15.62:0/3327395864 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8031960 con 0x7fe0e8154fa0 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 -- 172.21.15.62:0/3327395864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8154fa0 msgr2=0x7fe0e815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/3327395864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815e650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80cfeb0 tx=0x7fe0d802b840 comp rx=0 tx=0).stop 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 -- 172.21.15.62:0/3327395864 shutdown_connections 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/3327395864 >> v2:172.21.15.81:3300/0 conn(0x7fe0e8071c30 0x7fe0e80aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/3327395864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815e650 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.009+0000 7fe0ecc00640 1 -- 172.21.15.62:0/3327395864 >> 172.21.15.62:0/3327395864 conn(0x7fe0e80671f0 msgr2=0x7fe0e806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.010+0000 7fe0ecc00640 1 -- 172.21.15.62:0/3327395864 shutdown_connections 2024-03-28T18:00:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.010+0000 7fe0ecc00640 1 -- 172.21.15.62:0/3327395864 wait complete. 2024-03-28T18:00:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.011+0000 7fe0ecc00640 1 Processor -- start 2024-03-28T18:00:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.011+0000 7fe0ecc00640 1 -- start start 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0e80b4240 0x7fe0e8153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80b46d0 con 0x7fe0e8071c30 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80ae660 con 0x7fe0e8154fa0 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.012+0000 7fe0ecc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8155380 con 0x7fe0e80b4240 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42690/0 (socket says 172.21.15.62:42690) 2024-03-28T18:00:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 -- 172.21.15.62:0/2962493036 learned_addr learned my addr 172.21.15.62:0/2962493036 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 -- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0e80b4240 msgr2=0x7fe0e8153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0e80b4240 0x7fe0e8153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 -- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0e8154fa0 msgr2=0x7fe0e815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.013+0000 7fe0e5e00640 1 -- 172.21.15.62:0/2962493036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e815d700 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0e5e00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004230 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d80043d0 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8003850 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e815d9e0 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.014+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e81f4ec0 con 0x7fe0e8071c30 2024-03-28T18:00:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.015+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e81f5310 con 0x7fe0e8071c30 2024-03-28T18:00:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.016+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe0d80039f0 con 0x7fe0e8071c30 2024-03-28T18:00:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.018+0000 7fe0de000640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 0x7fe0c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.018+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0d802f030 con 0x7fe0e8071c30 2024-03-28T18:00:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.018+0000 7fe0e5400640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 0x7fe0c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.020+0000 7fe0e5400640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 0x7fe0c806e090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80b1eb0 tx=0x7fe0d4001310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.022+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fe0e81f5310 con 0x7fe0e8071c30 2024-03-28T18:00:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.221+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0e8153780 con 0x7fe0c806bbd0 2024-03-28T18:00:23.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.239+0000 7fe0de000640 1 -- 172.21.15.62:0/2962493036 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7fe0d4007070 con 0x7fe0c806bbd0 2024-03-28T18:00:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:23.244 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.244+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 msgr2=0x7fe0c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.244+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 0x7fe0c806e090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80b1eb0 tx=0x7fe0d4001310 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.244+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 msgr2=0x7fe0e8159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 shutdown_connections 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fe0c806bbd0 0x7fe0c806e090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0e8154fa0 0x7fe0e815ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fe0e80b4240 0x7fe0e8153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 --2- 172.21.15.62:0/2962493036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e8071c30 0x7fe0e8159e20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 >> 172.21.15.62:0/2962493036 conn(0x7fe0e80671f0 msgr2=0x7fe0e80b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 shutdown_connections 2024-03-28T18:00:23.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.245+0000 7fe0ecc00640 1 -- 172.21.15.62:0/2962493036 wait complete. 2024-03-28T18:00:23.262 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-28T18:00:22.335925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3960832,"data_stored":2242797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001793"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569583+0000","last_change":"2024-03-28T18:00:16.569583+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569583+0000","last_undegraded":"2024-03-28T18:00:16.569583+0000","last_fullsized":"2024-03-28T18:00:16.569583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579654+0000","last_change":"2024-03-28T18:00:16.579654+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579654+0000","last_undegraded":"2024-03-28T18:00:16.579654+0000","last_fullsized":"2024-03-28T18:00:16.579654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561545+0000","last_change":"2024-03-28T18:00:16.561545+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561545+0000","last_undegraded":"2024-03-28T18:00:16.561545+0000","last_fullsized":"2024-03-28T18:00:16.561545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571390+0000","last_change":"2024-03-28T18:00:16.571390+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571390+0000","last_undegraded":"2024-03-28T18:00:16.571390+0000","last_fullsized":"2024-03-28T18:00:16.571390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557319+0000","last_change":"2024-03-28T18:00:16.557319+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557319+0000","last_undegraded":"2024-03-28T18:00:16.557319+0000","last_fullsized":"2024-03-28T18:00:16.557319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.580779+0000","last_change":"2024-03-28T18:00:16.580779+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.580779+0000","last_undegraded":"2024-03-28T18:00:16.580779+0000","last_fullsized":"2024-03-28T18:00:16.580779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568442+0000","last_change":"2024-03-28T18:00:16.568442+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568442+0000","last_undegraded":"2024-03-28T18:00:16.568442+0000","last_fullsized":"2024-03-28T18:00:16.568442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575897+0000","last_change":"2024-03-28T18:00:16.575897+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575897+0000","last_undegraded":"2024-03-28T18:00:16.575897+0000","last_fullsized":"2024-03-28T18:00:16.575897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576562+0000","last_change":"2024-03-28T18:00:16.576562+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576562+0000","last_undegraded":"2024-03-28T18:00:16.576562+0000","last_fullsized":"2024-03-28T18:00:16.576562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568412+0000","last_change":"2024-03-28T18:00:16.568412+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568412+0000","last_undegraded":"2024-03-28T18:00:16.568412+0000","last_fullsized":"2024-03-28T18:00:16.568412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569231+0000","last_change":"2024-03-28T18:00:16.569231+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569231+0000","last_undegraded":"2024-03-28T18:00:16.569231+0000","last_fullsized":"2024-03-28T18:00:16.569231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559639+0000","last_change":"2024-03-28T18:00:16.559639+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559639+0000","last_undegraded":"2024-03-28T18:00:16.559639+0000","last_fullsized":"2024-03-28T18:00:16.559639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556645+0000","last_change":"2024-03-28T18:00:16.556645+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556645+0000","last_undegraded":"2024-03-28T18:00:16.556645+0000","last_fullsized":"2024-03-28T18:00:16.556645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572147+0000","last_change":"2024-03-28T18:00:16.572147+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572147+0000","last_undegraded":"2024-03-28T18:00:16.572147+0000","last_fullsized":"2024-03-28T18:00:16.572147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557955+0000","last_change":"2024-03-28T18:00:16.557955+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557955+0000","last_undegraded":"2024-03-28T18:00:16.557955+0000","last_fullsized":"2024-03-28T18:00:16.557955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567619+0000","last_change":"2024-03-28T18:00:16.567619+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567619+0000","last_undegraded":"2024-03-28T18:00:16.567619+0000","last_fullsized":"2024-03-28T18:00:16.567619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567977+0000","last_change":"2024-03-28T18:00:16.567977+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567977+0000","last_undegraded":"2024-03-28T18:00:16.567977+0000","last_fullsized":"2024-03-28T18:00:16.567977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.577391+0000","last_change":"2024-03-28T18:00:16.577391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.577391+0000","last_undegraded":"2024-03-28T18:00:16.577391+0000","last_fullsized":"2024-03-28T18:00:16.577391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573104+0000","last_change":"2024-03-28T18:00:16.573104+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573104+0000","last_undegraded":"2024-03-28T18:00:16.573104+0000","last_fullsized":"2024-03-28T18:00:16.573104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555854+0000","last_change":"2024-03-28T18:00:16.555854+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555854+0000","last_undegraded":"2024-03-28T18:00:16.555854+0000","last_fullsized":"2024-03-28T18:00:16.555854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576380+0000","last_change":"2024-03-28T18:00:16.576380+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576380+0000","last_undegraded":"2024-03-28T18:00:16.576380+0000","last_fullsized":"2024-03-28T18:00:16.576380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571362+0000","last_change":"2024-03-28T18:00:16.571362+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571362+0000","last_undegraded":"2024-03-28T18:00:16.571362+0000","last_fullsized":"2024-03-28T18:00:16.571362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561762+0000","last_change":"2024-03-28T18:00:16.561762+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561762+0000","last_undegraded":"2024-03-28T18:00:16.561762+0000","last_fullsized":"2024-03-28T18:00:16.561762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558028+0000","last_change":"2024-03-28T18:00:16.558028+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558028+0000","last_undegraded":"2024-03-28T18:00:16.558028+0000","last_fullsized":"2024-03-28T18:00:16.558028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569204+0000","last_change":"2024-03-28T18:00:16.569204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569204+0000","last_undegraded":"2024-03-28T18:00:16.569204+0000","last_fullsized":"2024-03-28T18:00:16.569204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576919+0000","last_change":"2024-03-28T18:00:16.576919+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576919+0000","last_undegraded":"2024-03-28T18:00:16.576919+0000","last_fullsized":"2024-03-28T18:00:16.576919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566641+0000","last_change":"2024-03-28T18:00:16.566641+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566641+0000","last_undegraded":"2024-03-28T18:00:16.566641+0000","last_fullsized":"2024-03-28T18:00:16.566641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558269+0000","last_change":"2024-03-28T18:00:16.558269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558269+0000","last_undegraded":"2024-03-28T18:00:16.558269+0000","last_fullsized":"2024-03-28T18:00:16.558269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575581+0000","last_change":"2024-03-28T18:00:16.575581+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575581+0000","last_undegraded":"2024-03-28T18:00:16.575581+0000","last_fullsized":"2024-03-28T18:00:16.575581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572204+0000","last_change":"2024-03-28T18:00:16.572204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572204+0000","last_undegraded":"2024-03-28T18:00:16.572204+0000","last_fullsized":"2024-03-28T18:00:16.572204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571667+0000","last_change":"2024-03-28T18:00:16.571667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571667+0000","last_undegraded":"2024-03-28T18:00:16.571667+0000","last_fullsized":"2024-03-28T18:00:16.571667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574076+0000","last_change":"2024-03-28T18:00:16.574076+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574076+0000","last_undegraded":"2024-03-28T18:00:16.574076+0000","last_fullsized":"2024-03-28T18:00:16.574076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562017+0000","last_change":"2024-03-28T18:00:16.562017+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562017+0000","last_undegraded":"2024-03-28T18:00:16.562017+0000","last_fullsized":"2024-03-28T18:00:16.562017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588360+0000","last_change":"2024-03-28T18:00:16.586041+0000","last_active":"2024-03-28T18:00:16.588360+0000","last_peered":"2024-03-28T18:00:16.588360+0000","last_clean":"2024-03-28T18:00:16.588360+0000","last_became_active":"2024-03-28T18:00:16.585654+0000","last_became_peered":"2024-03-28T18:00:16.585654+0000","last_unstale":"2024-03-28T18:00:16.588360+0000","last_undegraded":"2024-03-28T18:00:16.588360+0000","last_fullsized":"2024-03-28T18:00:16.588360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567054+0000","last_change":"2024-03-28T18:00:16.567054+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567054+0000","last_undegraded":"2024-03-28T18:00:16.567054+0000","last_fullsized":"2024-03-28T18:00:16.567054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586926+0000","last_change":"2024-03-28T18:00:16.582651+0000","last_active":"2024-03-28T18:00:16.586926+0000","last_peered":"2024-03-28T18:00:16.586926+0000","last_clean":"2024-03-28T18:00:16.586926+0000","last_became_active":"2024-03-28T18:00:16.582248+0000","last_became_peered":"2024-03-28T18:00:16.582248+0000","last_unstale":"2024-03-28T18:00:16.586926+0000","last_undegraded":"2024-03-28T18:00:16.586926+0000","last_fullsized":"2024-03-28T18:00:16.586926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571368+0000","last_change":"2024-03-28T18:00:16.571368+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571368+0000","last_undegraded":"2024-03-28T18:00:16.571368+0000","last_fullsized":"2024-03-28T18:00:16.571368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592321+0000","last_change":"2024-03-28T18:00:16.588264+0000","last_active":"2024-03-28T18:00:16.592321+0000","last_peered":"2024-03-28T18:00:16.592321+0000","last_clean":"2024-03-28T18:00:16.592321+0000","last_became_active":"2024-03-28T18:00:16.587925+0000","last_became_peered":"2024-03-28T18:00:16.587925+0000","last_unstale":"2024-03-28T18:00:16.592321+0000","last_undegraded":"2024-03-28T18:00:16.592321+0000","last_fullsized":"2024-03-28T18:00:16.592321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565721+0000","last_change":"2024-03-28T18:00:16.565721+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565721+0000","last_undegraded":"2024-03-28T18:00:16.565721+0000","last_fullsized":"2024-03-28T18:00:16.565721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579273+0000","last_change":"2024-03-28T18:00:16.577746+0000","last_active":"2024-03-28T18:00:16.579273+0000","last_peered":"2024-03-28T18:00:16.579273+0000","last_clean":"2024-03-28T18:00:16.579273+0000","last_became_active":"2024-03-28T18:00:16.577340+0000","last_became_peered":"2024-03-28T18:00:16.577340+0000","last_unstale":"2024-03-28T18:00:16.579273+0000","last_undegraded":"2024-03-28T18:00:16.579273+0000","last_fullsized":"2024-03-28T18:00:16.579273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570787+0000","last_change":"2024-03-28T18:00:16.570787+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570787+0000","last_undegraded":"2024-03-28T18:00:16.570787+0000","last_fullsized":"2024-03-28T18:00:16.570787+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593331+0000","last_change":"2024-03-28T18:00:16.591523+0000","last_active":"2024-03-28T18:00:16.593331+0000","last_peered":"2024-03-28T18:00:16.593331+0000","last_clean":"2024-03-28T18:00:16.593331+0000","last_became_active":"2024-03-28T18:00:16.591203+0000","last_became_peered":"2024-03-28T18:00:16.591203+0000","last_unstale":"2024-03-28T18:00:16.593331+0000","last_undegraded":"2024-03-28T18:00:16.593331+0000","last_fullsized":"2024-03-28T18:00:16.593331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:13.152406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559660+0000","last_change":"2024-03-28T18:00:16.559660+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559660+0000","last_undegraded":"2024-03-28T18:00:16.559660+0000","last_fullsized":"2024-03-28T18:00:16.559660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593355+0000","last_change":"2024-03-28T18:00:16.591608+0000","last_active":"2024-03-28T18:00:16.593355+0000","last_peered":"2024-03-28T18:00:16.593355+0000","last_clean":"2024-03-28T18:00:16.593355+0000","last_became_active":"2024-03-28T18:00:16.591289+0000","last_became_peered":"2024-03-28T18:00:16.591289+0000","last_unstale":"2024-03-28T18:00:16.593355+0000","last_undegraded":"2024-03-28T18:00:16.593355+0000","last_fullsized":"2024-03-28T18:00:16.593355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:23.345799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574763+0000","last_change":"2024-03-28T18:00:16.574763+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574763+0000","last_undegraded":"2024-03-28T18:00:16.574763+0000","last_fullsized":"2024-03-28T18:00:16.574763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589450+0000","last_change":"2024-03-28T18:00:16.581766+0000","last_active":"2024-03-28T18:00:16.589450+0000","last_peered":"2024-03-28T18:00:16.589450+0000","last_clean":"2024-03-28T18:00:16.589450+0000","last_became_active":"2024-03-28T18:00:16.581506+0000","last_became_peered":"2024-03-28T18:00:16.581506+0000","last_unstale":"2024-03-28T18:00:16.589450+0000","last_undegraded":"2024-03-28T18:00:16.589450+0000","last_fullsized":"2024-03-28T18:00:16.589450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.564269+0000","last_change":"2024-03-28T18:00:16.564269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.564269+0000","last_undegraded":"2024-03-28T18:00:16.564269+0000","last_fullsized":"2024-03-28T18:00:16.564269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590746+0000","last_change":"2024-03-28T18:00:16.589309+0000","last_active":"2024-03-28T18:00:16.590746+0000","last_peered":"2024-03-28T18:00:16.590746+0000","last_clean":"2024-03-28T18:00:16.590746+0000","last_became_active":"2024-03-28T18:00:16.588995+0000","last_became_peered":"2024-03-28T18:00:16.588995+0000","last_unstale":"2024-03-28T18:00:16.590746+0000","last_undegraded":"2024-03-28T18:00:16.590746+0000","last_fullsized":"2024-03-28T18:00:16.590746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:10.952085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560002+0000","last_change":"2024-03-28T18:00:16.560002+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560002+0000","last_undegraded":"2024-03-28T18:00:16.560002+0000","last_fullsized":"2024-03-28T18:00:16.560002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592334+0000","last_change":"2024-03-28T18:00:16.588159+0000","last_active":"2024-03-28T18:00:16.592334+0000","last_peered":"2024-03-28T18:00:16.592334+0000","last_clean":"2024-03-28T18:00:16.592334+0000","last_became_active":"2024-03-28T18:00:16.587891+0000","last_became_peered":"2024-03-28T18:00:16.587891+0000","last_unstale":"2024-03-28T18:00:16.592334+0000","last_undegraded":"2024-03-28T18:00:16.592334+0000","last_fullsized":"2024-03-28T18:00:16.592334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574440+0000","last_change":"2024-03-28T18:00:16.574440+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574440+0000","last_undegraded":"2024-03-28T18:00:16.574440+0000","last_fullsized":"2024-03-28T18:00:16.574440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592843+0000","last_change":"2024-03-28T18:00:16.590209+0000","last_active":"2024-03-28T18:00:16.592843+0000","last_peered":"2024-03-28T18:00:16.592843+0000","last_clean":"2024-03-28T18:00:16.592843+0000","last_became_active":"2024-03-28T18:00:16.589874+0000","last_became_peered":"2024-03-28T18:00:16.589874+0000","last_unstale":"2024-03-28T18:00:16.592843+0000","last_undegraded":"2024-03-28T18:00:16.592843+0000","last_fullsized":"2024-03-28T18:00:16.592843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573906+0000","last_change":"2024-03-28T18:00:16.573906+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573906+0000","last_undegraded":"2024-03-28T18:00:16.573906+0000","last_fullsized":"2024-03-28T18:00:16.573906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588152+0000","last_change":"2024-03-28T18:00:16.583925+0000","last_active":"2024-03-28T18:00:16.588152+0000","last_peered":"2024-03-28T18:00:16.588152+0000","last_clean":"2024-03-28T18:00:16.588152+0000","last_became_active":"2024-03-28T18:00:16.583635+0000","last_became_peered":"2024-03-28T18:00:16.583635+0000","last_unstale":"2024-03-28T18:00:16.588152+0000","last_undegraded":"2024-03-28T18:00:16.588152+0000","last_fullsized":"2024-03-28T18:00:16.588152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562694+0000","last_change":"2024-03-28T18:00:16.562694+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562694+0000","last_undegraded":"2024-03-28T18:00:16.562694+0000","last_fullsized":"2024-03-28T18:00:16.562694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.597940+0000","last_change":"2024-03-28T18:00:16.593104+0000","last_active":"2024-03-28T18:00:16.597940+0000","last_peered":"2024-03-28T18:00:16.597940+0000","last_clean":"2024-03-28T18:00:16.597940+0000","last_became_active":"2024-03-28T18:00:16.592771+0000","last_became_peered":"2024-03-28T18:00:16.592771+0000","last_unstale":"2024-03-28T18:00:16.597940+0000","last_undegraded":"2024-03-28T18:00:16.597940+0000","last_fullsized":"2024-03-28T18:00:16.597940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565690+0000","last_change":"2024-03-28T18:00:16.565690+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565690+0000","last_undegraded":"2024-03-28T18:00:16.565690+0000","last_fullsized":"2024-03-28T18:00:16.565690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589009+0000","last_change":"2024-03-28T18:00:16.584849+0000","last_active":"2024-03-28T18:00:16.589009+0000","last_peered":"2024-03-28T18:00:16.589009+0000","last_clean":"2024-03-28T18:00:16.589009+0000","last_became_active":"2024-03-28T18:00:16.584543+0000","last_became_peered":"2024-03-28T18:00:16.584543+0000","last_unstale":"2024-03-28T18:00:16.589009+0000","last_undegraded":"2024-03-28T18:00:16.589009+0000","last_fullsized":"2024-03-28T18:00:16.589009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:28:00.623248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579339+0000","last_change":"2024-03-28T18:00:16.579339+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579339+0000","last_undegraded":"2024-03-28T18:00:16.579339+0000","last_fullsized":"2024-03-28T18:00:16.579339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583382+0000","last_change":"2024-03-28T18:00:16.580448+0000","last_active":"2024-03-28T18:00:16.583382+0000","last_peered":"2024-03-28T18:00:16.583382+0000","last_clean":"2024-03-28T18:00:16.583382+0000","last_became_active":"2024-03-28T18:00:16.579745+0000","last_became_peered":"2024-03-28T18:00:16.579745+0000","last_unstale":"2024-03-28T18:00:16.583382+0000","last_undegraded":"2024-03-28T18:00:16.583382+0000","last_fullsized":"2024-03-28T18:00:16.583382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571904+0000","last_change":"2024-03-28T18:00:16.571904+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571904+0000","last_undegraded":"2024-03-28T18:00:16.571904+0000","last_fullsized":"2024-03-28T18:00:16.571904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592117+0000","last_change":"2024-03-28T18:00:16.587497+0000","last_active":"2024-03-28T18:00:16.592117+0000","last_peered":"2024-03-28T18:00:16.592117+0000","last_clean":"2024-03-28T18:00:16.592117+0000","last_became_active":"2024-03-28T18:00:16.586180+0000","last_became_peered":"2024-03-28T18:00:16.586180+0000","last_unstale":"2024-03-28T18:00:16.592117+0000","last_undegraded":"2024-03-28T18:00:16.592117+0000","last_fullsized":"2024-03-28T18:00:16.592117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566667+0000","last_change":"2024-03-28T18:00:16.566667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566667+0000","last_undegraded":"2024-03-28T18:00:16.566667+0000","last_fullsized":"2024-03-28T18:00:16.566667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589771+0000","last_change":"2024-03-28T18:00:16.581297+0000","last_active":"2024-03-28T18:00:16.589771+0000","last_peered":"2024-03-28T18:00:16.589771+0000","last_clean":"2024-03-28T18:00:16.589771+0000","last_became_active":"2024-03-28T18:00:16.580937+0000","last_became_peered":"2024-03-28T18:00:16.580937+0000","last_unstale":"2024-03-28T18:00:16.589771+0000","last_undegraded":"2024-03-28T18:00:16.589771+0000","last_fullsized":"2024-03-28T18:00:16.589771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571676+0000","last_change":"2024-03-28T18:00:16.571676+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571676+0000","last_undegraded":"2024-03-28T18:00:16.571676+0000","last_fullsized":"2024-03-28T18:00:16.571676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592286+0000","last_change":"2024-03-28T18:00:16.589262+0000","last_active":"2024-03-28T18:00:16.592286+0000","last_peered":"2024-03-28T18:00:16.592286+0000","last_clean":"2024-03-28T18:00:16.592286+0000","last_became_active":"2024-03-28T18:00:16.588786+0000","last_became_peered":"2024-03-28T18:00:16.588786+0000","last_unstale":"2024-03-28T18:00:16.592286+0000","last_undegraded":"2024-03-28T18:00:16.592286+0000","last_fullsized":"2024-03-28T18:00:16.592286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561624+0000","last_change":"2024-03-28T18:00:16.561624+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561624+0000","last_undegraded":"2024-03-28T18:00:16.561624+0000","last_fullsized":"2024-03-28T18:00:16.561624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589122+0000","last_change":"2024-03-28T18:00:16.587380+0000","last_active":"2024-03-28T18:00:16.589122+0000","last_peered":"2024-03-28T18:00:16.589122+0000","last_clean":"2024-03-28T18:00:16.589122+0000","last_became_active":"2024-03-28T18:00:16.586842+0000","last_became_peered":"2024-03-28T18:00:16.586842+0000","last_unstale":"2024-03-28T18:00:16.589122+0000","last_undegraded":"2024-03-28T18:00:16.589122+0000","last_fullsized":"2024-03-28T18:00:16.589122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.737137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570537+0000","last_change":"2024-03-28T18:00:16.570537+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570537+0000","last_undegraded":"2024-03-28T18:00:16.570537+0000","last_fullsized":"2024-03-28T18:00:16.570537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588036+0000","last_change":"2024-03-28T18:00:16.581961+0000","last_active":"2024-03-28T18:00:16.588036+0000","last_peered":"2024-03-28T18:00:16.588036+0000","last_clean":"2024-03-28T18:00:16.588036+0000","last_became_active":"2024-03-28T18:00:16.581694+0000","last_became_peered":"2024-03-28T18:00:16.581694+0000","last_unstale":"2024-03-28T18:00:16.588036+0000","last_undegraded":"2024-03-28T18:00:16.588036+0000","last_fullsized":"2024-03-28T18:00:16.588036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:58:38.567836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556811+0000","last_change":"2024-03-28T18:00:16.556811+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556811+0000","last_undegraded":"2024-03-28T18:00:16.556811+0000","last_fullsized":"2024-03-28T18:00:16.556811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.591372+0000","last_change":"2024-03-28T18:00:16.590452+0000","last_active":"2024-03-28T18:00:16.591372+0000","last_peered":"2024-03-28T18:00:16.591372+0000","last_clean":"2024-03-28T18:00:16.591372+0000","last_became_active":"2024-03-28T18:00:16.590148+0000","last_became_peered":"2024-03-28T18:00:16.590148+0000","last_unstale":"2024-03-28T18:00:16.591372+0000","last_undegraded":"2024-03-28T18:00:16.591372+0000","last_fullsized":"2024-03-28T18:00:16.591372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:59.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555930+0000","last_change":"2024-03-28T18:00:16.555930+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555930+0000","last_undegraded":"2024-03-28T18:00:16.555930+0000","last_fullsized":"2024-03-28T18:00:16.555930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.578677+0000","last_change":"2024-03-28T18:00:16.574301+0000","last_active":"2024-03-28T18:00:16.578677+0000","last_peered":"2024-03-28T18:00:16.578677+0000","last_clean":"2024-03-28T18:00:16.578677+0000","last_became_active":"2024-03-28T18:00:16.573671+0000","last_became_peered":"2024-03-28T18:00:16.573671+0000","last_unstale":"2024-03-28T18:00:16.578677+0000","last_undegraded":"2024-03-28T18:00:16.578677+0000","last_fullsized":"2024-03-28T18:00:16.578677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555199+0000","last_change":"2024-03-28T18:00:16.555199+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555199+0000","last_undegraded":"2024-03-28T18:00:16.555199+0000","last_fullsized":"2024-03-28T18:00:16.555199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586129+0000","last_change":"2024-03-28T18:00:16.584421+0000","last_active":"2024-03-28T18:00:16.586129+0000","last_peered":"2024-03-28T18:00:16.586129+0000","last_clean":"2024-03-28T18:00:16.586129+0000","last_became_active":"2024-03-28T18:00:16.584113+0000","last_became_peered":"2024-03-28T18:00:16.584113+0000","last_unstale":"2024-03-28T18:00:16.586129+0000","last_undegraded":"2024-03-28T18:00:16.586129+0000","last_fullsized":"2024-03-28T18:00:16.586129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556334+0000","last_change":"2024-03-28T18:00:16.556334+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556334+0000","last_undegraded":"2024-03-28T18:00:16.556334+0000","last_fullsized":"2024-03-28T18:00:16.556334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.582176+0000","last_change":"2024-03-28T18:00:16.578260+0000","last_active":"2024-03-28T18:00:16.582176+0000","last_peered":"2024-03-28T18:00:16.582176+0000","last_clean":"2024-03-28T18:00:16.582176+0000","last_became_active":"2024-03-28T18:00:16.577824+0000","last_became_peered":"2024-03-28T18:00:16.577824+0000","last_unstale":"2024-03-28T18:00:16.582176+0000","last_undegraded":"2024-03-28T18:00:16.582176+0000","last_fullsized":"2024-03-28T18:00:16.582176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569631+0000","last_change":"2024-03-28T18:00:16.569631+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569631+0000","last_undegraded":"2024-03-28T18:00:16.569631+0000","last_fullsized":"2024-03-28T18:00:16.569631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588830+0000","last_change":"2024-03-28T18:00:16.584736+0000","last_active":"2024-03-28T18:00:16.588830+0000","last_peered":"2024-03-28T18:00:16.588830+0000","last_clean":"2024-03-28T18:00:16.588830+0000","last_became_active":"2024-03-28T18:00:16.584455+0000","last_became_peered":"2024-03-28T18:00:16.584455+0000","last_unstale":"2024-03-28T18:00:16.588830+0000","last_undegraded":"2024-03-28T18:00:16.588830+0000","last_fullsized":"2024-03-28T18:00:16.588830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557269+0000","last_change":"2024-03-28T18:00:16.557269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557269+0000","last_undegraded":"2024-03-28T18:00:16.557269+0000","last_fullsized":"2024-03-28T18:00:16.557269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.584256+0000","last_change":"2024-03-28T18:00:16.583053+0000","last_active":"2024-03-28T18:00:16.584256+0000","last_peered":"2024-03-28T18:00:16.584256+0000","last_clean":"2024-03-28T18:00:16.584256+0000","last_became_active":"2024-03-28T18:00:16.582775+0000","last_became_peered":"2024-03-28T18:00:16.582775+0000","last_unstale":"2024-03-28T18:00:16.584256+0000","last_undegraded":"2024-03-28T18:00:16.584256+0000","last_fullsized":"2024-03-28T18:00:16.584256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560345+0000","last_change":"2024-03-28T18:00:16.560345+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560345+0000","last_undegraded":"2024-03-28T18:00:16.560345+0000","last_fullsized":"2024-03-28T18:00:16.560345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579729+0000","last_change":"2024-03-28T18:00:16.575792+0000","last_active":"2024-03-28T18:00:16.579729+0000","last_peered":"2024-03-28T18:00:16.579729+0000","last_clean":"2024-03-28T18:00:16.579729+0000","last_became_active":"2024-03-28T18:00:16.575475+0000","last_became_peered":"2024-03-28T18:00:16.575475+0000","last_unstale":"2024-03-28T18:00:16.579729+0000","last_undegraded":"2024-03-28T18:00:16.579729+0000","last_fullsized":"2024-03-28T18:00:16.579729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562722+0000","last_change":"2024-03-28T18:00:16.562722+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562722+0000","last_undegraded":"2024-03-28T18:00:16.562722+0000","last_fullsized":"2024-03-28T18:00:16.562722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579125+0000","last_change":"2024-03-28T18:00:16.576446+0000","last_active":"2024-03-28T18:00:16.579125+0000","last_peered":"2024-03-28T18:00:16.579125+0000","last_clean":"2024-03-28T18:00:16.579125+0000","last_became_active":"2024-03-28T18:00:16.575444+0000","last_became_peered":"2024-03-28T18:00:16.575444+0000","last_unstale":"2024-03-28T18:00:16.579125+0000","last_undegraded":"2024-03-28T18:00:16.579125+0000","last_fullsized":"2024-03-28T18:00:16.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567784+0000","last_change":"2024-03-28T18:00:16.567784+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567784+0000","last_undegraded":"2024-03-28T18:00:16.567784+0000","last_fullsized":"2024-03-28T18:00:16.567784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.577769+0000","last_change":"2024-03-28T18:00:16.573496+0000","last_active":"2024-03-28T18:00:16.577769+0000","last_peered":"2024-03-28T18:00:16.577769+0000","last_clean":"2024-03-28T18:00:16.577769+0000","last_became_active":"2024-03-28T18:00:16.573041+0000","last_became_peered":"2024-03-28T18:00:16.573041+0000","last_unstale":"2024-03-28T18:00:16.577769+0000","last_undegraded":"2024-03-28T18:00:16.577769+0000","last_fullsized":"2024-03-28T18:00:16.577769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:56.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561746+0000","last_change":"2024-03-28T18:00:16.561746+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561746+0000","last_undegraded":"2024-03-28T18:00:16.561746+0000","last_fullsized":"2024-03-28T18:00:16.561746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583640+0000","last_change":"2024-03-28T18:00:16.581384+0000","last_active":"2024-03-28T18:00:16.583640+0000","last_peered":"2024-03-28T18:00:16.583640+0000","last_clean":"2024-03-28T18:00:16.583640+0000","last_became_active":"2024-03-28T18:00:16.581030+0000","last_became_peered":"2024-03-28T18:00:16.581030+0000","last_unstale":"2024-03-28T18:00:16.583640+0000","last_undegraded":"2024-03-28T18:00:16.583640+0000","last_fullsized":"2024-03-28T18:00:16.583640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569391+0000","last_change":"2024-03-28T18:00:16.569391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569391+0000","last_undegraded":"2024-03-28T18:00:16.569391+0000","last_fullsized":"2024-03-28T18:00:16.569391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.587915+0000","last_change":"2024-03-28T18:00:16.582503+0000","last_active":"2024-03-28T18:00:16.587915+0000","last_peered":"2024-03-28T18:00:16.587915+0000","last_clean":"2024-03-28T18:00:16.587915+0000","last_became_active":"2024-03-28T18:00:16.581354+0000","last_became_peered":"2024-03-28T18:00:16.581354+0000","last_unstale":"2024-03-28T18:00:16.587915+0000","last_undegraded":"2024-03-28T18:00:16.587915+0000","last_fullsized":"2024-03-28T18:00:16.587915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:02.785011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575167+0000","last_change":"2024-03-28T18:00:16.575167+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575167+0000","last_undegraded":"2024-03-28T18:00:16.575167+0000","last_fullsized":"2024-03-28T18:00:16.575167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592189+0000","last_change":"2024-03-28T18:00:16.587890+0000","last_active":"2024-03-28T18:00:16.592189+0000","last_peered":"2024-03-28T18:00:16.592189+0000","last_clean":"2024-03-28T18:00:16.592189+0000","last_became_active":"2024-03-28T18:00:16.587564+0000","last_became_peered":"2024-03-28T18:00:16.587564+0000","last_unstale":"2024-03-28T18:00:16.592189+0000","last_undegraded":"2024-03-28T18:00:16.592189+0000","last_fullsized":"2024-03-28T18:00:16.592189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576464+0000","last_change":"2024-03-28T18:00:16.576464+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576464+0000","last_undegraded":"2024-03-28T18:00:16.576464+0000","last_fullsized":"2024-03-28T18:00:16.576464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590990+0000","last_change":"2024-03-28T18:00:16.589804+0000","last_active":"2024-03-28T18:00:16.590990+0000","last_peered":"2024-03-28T18:00:16.590990+0000","last_clean":"2024-03-28T18:00:16.590990+0000","last_became_active":"2024-03-28T18:00:16.589477+0000","last_became_peered":"2024-03-28T18:00:16.589477+0000","last_unstale":"2024-03-28T18:00:16.590990+0000","last_undegraded":"2024-03-28T18:00:16.590990+0000","last_fullsized":"2024-03-28T18:00:16.590990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579800+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T18:00:16.579800+0000","last_peered":"2024-03-28T18:00:16.579800+0000","last_clean":"2024-03-28T18:00:16.579800+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T18:00:16.579800+0000","last_undegraded":"2024-03-28T18:00:16.579800+0000","last_fullsized":"2024-03-28T18:00:16.579800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.569583+0000', 'last_change': '2024-03-28T18:00:16.569583+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.569583+0000', 'last_undegraded': '2024-03-28T18:00:16.569583+0000', 'last_fullsized': '2024-03-28T18:00:16.569583+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.579654+0000', 'last_change': '2024-03-28T18:00:16.579654+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.579654+0000', 'last_undegraded': '2024-03-28T18:00:16.579654+0000', 'last_fullsized': '2024-03-28T18:00:16.579654+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.561545+0000', 'last_change': '2024-03-28T18:00:16.561545+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.561545+0000', 'last_undegraded': '2024-03-28T18:00:16.561545+0000', 'last_fullsized': '2024-03-28T18:00:16.561545+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571390+0000', 'last_change': '2024-03-28T18:00:16.571390+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571390+0000', 'last_undegraded': '2024-03-28T18:00:16.571390+0000', 'last_fullsized': '2024-03-28T18:00:16.571390+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-28T18:00:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.557319+0000', 'last_change': '2024-03-28T18:00:16.557319+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.557319+0000', 'last_undegraded': '2024-03-28T18:00:16.557319+0000', 'last_fullsized': '2024-03-28T18:00:16.557319+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.580779+0000', 'last_change': '2024-03-28T18:00:16.580779+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.580779+0000', 'last_undegraded': '2024-03-28T18:00:16.580779+0000', 'last_fullsized': '2024-03-28T18:00:16.580779+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.568442+0000', 'last_change': '2024-03-28T18:00:16.568442+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.568442+0000', 'last_undegraded': '2024-03-28T18:00:16.568442+0000', 'last_fullsized': '2024-03-28T18:00:16.568442+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.575897+0000', 'last_change': '2024-03-28T18:00:16.575897+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.575897+0000', 'last_undegraded': '2024-03-28T18:00:16.575897+0000', 'last_fullsized': '2024-03-28T18:00:16.575897+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.576562+0000', 'last_change': '2024-03-28T18:00:16.576562+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.576562+0000', 'last_undegraded': '2024-03-28T18:00:16.576562+0000', 'last_fullsized': '2024-03-28T18:00:16.576562+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.568412+0000', 'last_change': '2024-03-28T18:00:16.568412+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.568412+0000', 'last_undegraded': '2024-03-28T18:00:16.568412+0000', 'last_fullsized': '2024-03-28T18:00:16.568412+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.569231+0000', 'last_change': '2024-03-28T18:00:16.569231+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.569231+0000', 'last_undegraded': '2024-03-28T18:00:16.569231+0000', 'last_fullsized': '2024-03-28T18:00:16.569231+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.559639+0000', 'last_change': '2024-03-28T18:00:16.559639+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.559639+0000', 'last_undegraded': '2024-03-28T18:00:16.559639+0000', 'last_fullsized': '2024-03-28T18:00:16.559639+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.556645+0000', 'last_change': '2024-03-28T18:00:16.556645+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.556645+0000', 'last_undegraded': '2024-03-28T18:00:16.556645+0000', 'last_fullsized': '2024-03-28T18:00:16.556645+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.572147+0000', 'last_change': '2024-03-28T18:00:16.572147+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.572147+0000', 'last_undegraded': '2024-03-28T18:00:16.572147+0000', 'last_fullsized': '2024-03-28T18:00:16.572147+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.557955+0000', 'last_change': '2024-03-28T18:00:16.557955+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.557955+0000', 'last_undegraded': '2024-03-28T18:00:16.557955+0000', 'last_fullsized': '2024-03-28T18:00:16.557955+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.567619+0000', 'last_change': '2024-03-28T18:00:16.567619+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.567619+0000', 'last_undegraded': '2024-03-28T18:00:16.567619+0000', 'last_fullsized': '2024-03-28T18:00:16.567619+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.567977+0000', 'last_change': '2024-03-28T18:00:16.567977+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.567977+0000', 'last_undegraded': '2024-03-28T18:00:16.567977+0000', 'last_fullsized': '2024-03-28T18:00:16.567977+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.577391+0000', 'last_change': '2024-03-28T18:00:16.577391+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.577391+0000', 'last_undegraded': '2024-03-28T18:00:16.577391+0000', 'last_fullsized': '2024-03-28T18:00:16.577391+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.573104+0000', 'last_change': '2024-03-28T18:00:16.573104+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.573104+0000', 'last_undegraded': '2024-03-28T18:00:16.573104+0000', 'last_fullsized': '2024-03-28T18:00:16.573104+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.555854+0000', 'last_change': '2024-03-28T18:00:16.555854+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.555854+0000', 'last_undegraded': '2024-03-28T18:00:16.555854+0000', 'last_fullsized': '2024-03-28T18:00:16.555854+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.576380+0000', 'last_change': '2024-03-28T18:00:16.576380+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.576380+0000', 'last_undegraded': '2024-03-28T18:00:16.576380+0000', 'last_fullsized': '2024-03-28T18:00:16.576380+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571362+0000', 'last_change': '2024-03-28T18:00:16.571362+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571362+0000', 'last_undegraded': '2024-03-28T18:00:16.571362+0000', 'last_fullsized': '2024-03-28T18:00:16.571362+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.561762+0000', 'last_change': '2024-03-28T18:00:16.561762+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.561762+0000', 'last_undegraded': '2024-03-28T18:00:16.561762+0000', 'last_fullsized': '2024-03-28T18:00:16.561762+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.558028+0000', 'last_change': '2024-03-28T18:00:16.558028+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.558028+0000', 'last_undegraded': '2024-03-28T18:00:16.558028+0000', 'last_fullsized': '2024-03-28T18:00:16.558028+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.569204+0000', 'last_change': '2024-03-28T18:00:16.569204+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.569204+0000', 'last_undegraded': '2024-03-28T18:00:16.569204+0000', 'last_fullsized': '2024-03-28T18:00:16.569204+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.576919+0000', 'last_change': '2024-03-28T18:00:16.576919+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.576919+0000', 'last_undegraded': '2024-03-28T18:00:16.576919+0000', 'last_fullsized': '2024-03-28T18:00:16.576919+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.566641+0000', 'last_change': '2024-03-28T18:00:16.566641+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.566641+0000', 'last_undegraded': '2024-03-28T18:00:16.566641+0000', 'last_fullsized': '2024-03-28T18:00:16.566641+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-28T18:00:23.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.558269+0000', 'last_change': '2024-03-28T18:00:16.558269+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.558269+0000', 'last_undegraded': '2024-03-28T18:00:16.558269+0000', 'last_fullsized': '2024-03-28T18:00:16.558269+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.575581+0000', 'last_change': '2024-03-28T18:00:16.575581+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.575581+0000', 'last_undegraded': '2024-03-28T18:00:16.575581+0000', 'last_fullsized': '2024-03-28T18:00:16.575581+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.572204+0000', 'last_change': '2024-03-28T18:00:16.572204+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.572204+0000', 'last_undegraded': '2024-03-28T18:00:16.572204+0000', 'last_fullsized': '2024-03-28T18:00:16.572204+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571667+0000', 'last_change': '2024-03-28T18:00:16.571667+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571667+0000', 'last_undegraded': '2024-03-28T18:00:16.571667+0000', 'last_fullsized': '2024-03-28T18:00:16.571667+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.574076+0000', 'last_change': '2024-03-28T18:00:16.574076+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.574076+0000', 'last_undegraded': '2024-03-28T18:00:16.574076+0000', 'last_fullsized': '2024-03-28T18:00:16.574076+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.562017+0000', 'last_change': '2024-03-28T18:00:16.562017+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.562017+0000', 'last_undegraded': '2024-03-28T18:00:16.562017+0000', 'last_fullsized': '2024-03-28T18:00:16.562017+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-28T18:00:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.567054+0000', 'last_change': '2024-03-28T18:00:16.567054+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.567054+0000', 'last_undegraded': '2024-03-28T18:00:16.567054+0000', 'last_fullsized': '2024-03-28T18:00:16.567054+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571368+0000', 'last_change': '2024-03-28T18:00:16.571368+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571368+0000', 'last_undegraded': '2024-03-28T18:00:16.571368+0000', 'last_fullsized': '2024-03-28T18:00:16.571368+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.565721+0000', 'last_change': '2024-03-28T18:00:16.565721+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.565721+0000', 'last_undegraded': '2024-03-28T18:00:16.565721+0000', 'last_fullsized': '2024-03-28T18:00:16.565721+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.570787+0000', 'last_change': '2024-03-28T18:00:16.570787+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.570787+0000', 'last_undegraded': '2024-03-28T18:00:16.570787+0000', 'last_fullsized': '2024-03-28T18:00:16.570787+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.559660+0000', 'last_change': '2024-03-28T18:00:16.559660+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.559660+0000', 'last_undegraded': '2024-03-28T18:00:16.559660+0000', 'last_fullsized': '2024-03-28T18:00:16.559660+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-28T18:00:23.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.574763+0000', 'last_change': '2024-03-28T18:00:16.574763+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.574763+0000', 'last_undegraded': '2024-03-28T18:00:16.574763+0000', 'last_fullsized': '2024-03-28T18:00:16.574763+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.564269+0000', 'last_change': '2024-03-28T18:00:16.564269+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.564269+0000', 'last_undegraded': '2024-03-28T18:00:16.564269+0000', 'last_fullsized': '2024-03-28T18:00:16.564269+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.560002+0000', 'last_change': '2024-03-28T18:00:16.560002+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.560002+0000', 'last_undegraded': '2024-03-28T18:00:16.560002+0000', 'last_fullsized': '2024-03-28T18:00:16.560002+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.574440+0000', 'last_change': '2024-03-28T18:00:16.574440+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.574440+0000', 'last_undegraded': '2024-03-28T18:00:16.574440+0000', 'last_fullsized': '2024-03-28T18:00:16.574440+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.573906+0000', 'last_change': '2024-03-28T18:00:16.573906+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.573906+0000', 'last_undegraded': '2024-03-28T18:00:16.573906+0000', 'last_fullsized': '2024-03-28T18:00:16.573906+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.562694+0000', 'last_change': '2024-03-28T18:00:16.562694+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.562694+0000', 'last_undegraded': '2024-03-28T18:00:16.562694+0000', 'last_fullsized': '2024-03-28T18:00:16.562694+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.565690+0000', 'last_change': '2024-03-28T18:00:16.565690+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.565690+0000', 'last_undegraded': '2024-03-28T18:00:16.565690+0000', 'last_fullsized': '2024-03-28T18:00:16.565690+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.579339+0000', 'last_change': '2024-03-28T18:00:16.579339+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.579339+0000', 'last_undegraded': '2024-03-28T18:00:16.579339+0000', 'last_fullsized': '2024-03-28T18:00:16.579339+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571904+0000', 'last_change': '2024-03-28T18:00:16.571904+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571904+0000', 'last_undegraded': '2024-03-28T18:00:16.571904+0000', 'last_fullsized': '2024-03-28T18:00:16.571904+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.566667+0000', 'last_change': '2024-03-28T18:00:16.566667+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.566667+0000', 'last_undegraded': '2024-03-28T18:00:16.566667+0000', 'last_fullsized': '2024-03-28T18:00:16.566667+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.571676+0000', 'last_change': '2024-03-28T18:00:16.571676+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.571676+0000', 'last_undegraded': '2024-03-28T18:00:16.571676+0000', 'last_fullsized': '2024-03-28T18:00:16.571676+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.561624+0000', 'last_change': '2024-03-28T18:00:16.561624+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.561624+0000', 'last_undegraded': '2024-03-28T18:00:16.561624+0000', 'last_fullsized': '2024-03-28T18:00:16.561624+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-28T18:00:23.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.570537+0000', 'last_change': '2024-03-28T18:00:16.570537+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.570537+0000', 'last_undegraded': '2024-03-28T18:00:16.570537+0000', 'last_fullsized': '2024-03-28T18:00:16.570537+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.556811+0000', 'last_change': '2024-03-28T18:00:16.556811+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.556811+0000', 'last_undegraded': '2024-03-28T18:00:16.556811+0000', 'last_fullsized': '2024-03-28T18:00:16.556811+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.555930+0000', 'last_change': '2024-03-28T18:00:16.555930+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.555930+0000', 'last_undegraded': '2024-03-28T18:00:16.555930+0000', 'last_fullsized': '2024-03-28T18:00:16.555930+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.555199+0000', 'last_change': '2024-03-28T18:00:16.555199+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.555199+0000', 'last_undegraded': '2024-03-28T18:00:16.555199+0000', 'last_fullsized': '2024-03-28T18:00:16.555199+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.556334+0000', 'last_change': '2024-03-28T18:00:16.556334+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.556334+0000', 'last_undegraded': '2024-03-28T18:00:16.556334+0000', 'last_fullsized': '2024-03-28T18:00:16.556334+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.569631+0000', 'last_change': '2024-03-28T18:00:16.569631+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.569631+0000', 'last_undegraded': '2024-03-28T18:00:16.569631+0000', 'last_fullsized': '2024-03-28T18:00:16.569631+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.557269+0000', 'last_change': '2024-03-28T18:00:16.557269+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.557269+0000', 'last_undegraded': '2024-03-28T18:00:16.557269+0000', 'last_fullsized': '2024-03-28T18:00:16.557269+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.560345+0000', 'last_change': '2024-03-28T18:00:16.560345+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.560345+0000', 'last_undegraded': '2024-03-28T18:00:16.560345+0000', 'last_fullsized': '2024-03-28T18:00:16.560345+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.562722+0000', 'last_change': '2024-03-28T18:00:16.562722+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.562722+0000', 'last_undegraded': '2024-03-28T18:00:16.562722+0000', 'last_fullsized': '2024-03-28T18:00:16.562722+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.567784+0000', 'last_change': '2024-03-28T18:00:16.567784+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.567784+0000', 'last_undegraded': '2024-03-28T18:00:16.567784+0000', 'last_fullsized': '2024-03-28T18:00:16.567784+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.561746+0000', 'last_change': '2024-03-28T18:00:16.561746+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.561746+0000', 'last_undegraded': '2024-03-28T18:00:16.561746+0000', 'last_fullsized': '2024-03-28T18:00:16.561746+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-28T18:00:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.569391+0000', 'last_change': '2024-03-28T18:00:16.569391+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.569391+0000', 'last_undegraded': '2024-03-28T18:00:16.569391+0000', 'last_fullsized': '2024-03-28T18:00:16.569391+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-28T18:00:23.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-28T18:00:23.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.575167+0000', 'last_change': '2024-03-28T18:00:16.575167+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.575167+0000', 'last_undegraded': '2024-03-28T18:00:16.575167+0000', 'last_fullsized': '2024-03-28T18:00:16.575167+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-28T18:00:23.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-28T18:00:23.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-28T18:00:16.576464+0000', 'last_change': '2024-03-28T18:00:16.576464+0000', 'last_active': '2024-03-28T18:00:16.533783+0000', 'last_peered': '2024-03-28T18:00:16.533783+0000', 'last_clean': '2024-03-28T18:00:16.533783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-28T18:00:16.576464+0000', 'last_undegraded': '2024-03-28T18:00:16.576464+0000', 'last_fullsized': '2024-03-28T18:00:16.576464+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'last_clean_scrub_stamp': '2024-03-28T18:00:16.533783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-28T18:00:23.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-28T18:00:23.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.518+0000 7f6c80a00640 1 Processor -- start 2024-03-28T18:00:23.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.519+0000 7f6c80a00640 1 -- start start 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c1594e0 0x7f6c7c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c7c0aec60 0x7f6c7c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c7c15c8d0 0x7f6c7c070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0af0c0 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b2800 con 0x7f6c7c0aec60 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.520+0000 7f6c80a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b2a50 con 0x7f6c7c1594e0 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c7c15c8d0 0x7f6c7c070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c7c15c8d0 0x7f6c7c070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42700/0 (socket says 172.21.15.62:42700) 2024-03-28T18:00:23.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 -- 172.21.15.62:0/3005961558 learned_addr learned my addr 172.21.15.62:0/3005961558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7a800640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c1594e0 0x7f6c7c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c73e00640 1 --2- 172.21.15.62:0/3005961558 >> v2:172.21.15.81:3300/0 conn(0x7f6c7c0aec60 0x7f6c7c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 -- 172.21.15.62:0/3005961558 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c1594e0 msgr2=0x7f6c7c0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c1594e0 0x7f6c7c0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 -- 172.21.15.62:0/3005961558 >> v2:172.21.15.81:3300/0 conn(0x7f6c7c0aec60 msgr2=0x7f6c7c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 --2- 172.21.15.62:0/3005961558 >> v2:172.21.15.81:3300/0 conn(0x7f6c7c0aec60 0x7f6c7c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.521+0000 7f6c7b200640 1 -- 172.21.15.62:0/3005961558 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c0b2d30 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.523+0000 7f6c7b200640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c15c8d0 0x7f6c7c070890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0b2c60 tx=0x7f6c7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4002cd376fd3bf27 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.523+0000 7f6c79e00640 1 -- 172.21.15.62:0/3005961558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7402ca70 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.523+0000 7f6c79e00640 1 -- 172.21.15.62:0/3005961558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7402cc10 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.523+0000 7f6c79e00640 1 -- 172.21.15.62:0/3005961558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740343f0 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 -- 172.21.15.62:0/3005961558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c15c8d0 msgr2=0x7f6c7c070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c15c8d0 0x7f6c7c070890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0b2c60 tx=0x7f6c7402bed0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 -- 172.21.15.62:0/3005961558 shutdown_connections 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c15c8d0 0x7f6c7c070890 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 --2- 172.21.15.62:0/3005961558 >> v2:172.21.15.81:3300/0 conn(0x7f6c7c0aec60 0x7f6c7c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 --2- 172.21.15.62:0/3005961558 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c1594e0 0x7f6c7c0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 -- 172.21.15.62:0/3005961558 >> 172.21.15.62:0/3005961558 conn(0x7f6c7c066dd0 msgr2=0x7f6c7c0671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:23.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 -- 172.21.15.62:0/3005961558 shutdown_connections 2024-03-28T18:00:23.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.524+0000 7f6c80a00640 1 -- 172.21.15.62:0/3005961558 wait complete. 2024-03-28T18:00:23.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.527+0000 7f6c80a00640 1 Processor -- start 2024-03-28T18:00:23.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.527+0000 7f6c80a00640 1 -- start start 2024-03-28T18:00:23.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c7c1594e0 0x7f6c7c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c15c8d0 0x7f6c7c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b3810 con 0x7f6c7c0aec60 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b32f0 con 0x7f6c7c1594e0 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c80a00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0c4aa0 con 0x7f6c7c15c8d0 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42702/0 (socket says 172.21.15.62:42702) 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c7a800640 1 -- 172.21.15.62:0/487082616 learned_addr learned my addr 172.21.15.62:0/487082616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.529+0000 7f6c73e00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c7c1594e0 0x7f6c7c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7b200640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c15c8d0 0x7f6c7c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 -- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c15c8d0 msgr2=0x7f6c7c15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c15c8d0 0x7f6c7c15ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 -- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c7c1594e0 msgr2=0x7f6c7c14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c7c1594e0 0x7f6c7c14b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 -- 172.21.15.62:0/487082616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c1568a0 con 0x7f6c7c0aec60 2024-03-28T18:00:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c7a800640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d830 tx=0x7f6c6c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00b6c0 con 0x7f6c7c0aec60 2024-03-28T18:00:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c019070 con 0x7f6c7c0aec60 2024-03-28T18:00:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.530+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c156b80 con 0x7f6c7c0aec60 2024-03-28T18:00:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.531+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00ba80 con 0x7f6c7c0aec60 2024-03-28T18:00:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.531+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c14b720 con 0x7f6c7c0aec60 2024-03-28T18:00:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.532+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c04f400 con 0x7f6c7c0aec60 2024-03-28T18:00:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.533+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6c6c01e460 con 0x7f6c7c0aec60 2024-03-28T18:00:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.534+0000 7f6c72a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 0x7f6c5806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.535+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c6c08b0d0 con 0x7f6c7c0aec60 2024-03-28T18:00:23.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.535+0000 7f6c73e00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 0x7f6c5806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.536+0000 7f6c73e00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 0x7f6c5806e090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0cfa60 tx=0x7f6c64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:23.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.539+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f6c6c059bc0 con 0x7f6c7c0aec60 2024-03-28T18:00:23.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.874+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6c7c0ccd30 con 0x7f6c7c0aec60 2024-03-28T18:00:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.875+0000 7f6c72a00640 1 -- 172.21.15.62:0/487082616 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1034 (secure 0 0 0) 0x7f6c6c0594c0 con 0x7f6c7c0aec60 2024-03-28T18:00:23.875 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:23.875 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1711648721,"num_in_osds":8,"osd_in_since":1711648712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225320960,"bytes_avail":772868792320,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5208}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-28T18:00:00.329128+0000","services":{}},"progress_events":{}} 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.877+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 msgr2=0x7f6c5806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.877+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 0x7f6c5806e090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0cfa60 tx=0x7f6c64008040 comp rx=0 tx=0).stop 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.877+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 msgr2=0x7f6c7c157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.877+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d830 tx=0x7f6c6c00dcf0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 shutdown_connections 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f6c5806bbd0 0x7f6c5806e090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f6c7c15c8d0 0x7f6c7c15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c7c1594e0 0x7f6c7c14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 --2- 172.21.15.62:0/487082616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c7c0aec60 0x7f6c7c157380 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 >> 172.21.15.62:0/487082616 conn(0x7f6c7c066dd0 msgr2=0x7f6c7c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 shutdown_connections 2024-03-28T18:00:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:23.878+0000 7f6c80a00640 1 -- 172.21.15.62:0/487082616 wait complete. 2024-03-28T18:00:23.896 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-28T18:00:23.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:00:24.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.106+0000 7f76cbe00640 1 Processor -- start 2024-03-28T18:00:24.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.107+0000 7f76cbe00640 1 -- start start 2024-03-28T18:00:24.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.108+0000 7f76cbe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76c406d7f0 0x7f76c4071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.108+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76c40721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.108+0000 7f76cbe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c415ba70 con 0x7f76c4154b70 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.108+0000 7f76cbe00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c406d3c0 con 0x7f76c406d7f0 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.108+0000 7f76cbe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4154f50 con 0x7f76c406dc80 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c9400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f76c4154b70 0x7f76c406ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41454/0 (socket says 172.21.15.62:41454) 2024-03-28T18:00:24.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c9400640 1 -- 172.21.15.62:0/857911227 learned_addr learned my addr 172.21.15.62:0/857911227 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c8a00640 1 --2- 172.21.15.62:0/857911227 >> v2:172.21.15.81:3300/0 conn(0x7f76c406d7f0 0x7f76c4071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c9e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76c40721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335627336 0 0) 0x7f76c415ba70 con 0x7f76c4154b70 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.109+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ac005730 con 0x7f76c4154b70 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1286095550 0 0) 0x7f76ac005730 con 0x7f76c4154b70 2024-03-28T18:00:24.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 msgr2=0x7f76c40721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76c40721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 >> v2:172.21.15.81:3300/0 conn(0x7f76c406d7f0 msgr2=0x7f76c4071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 --2- 172.21.15.62:0/857911227 >> v2:172.21.15.81:3300/0 conn(0x7f76c406d7f0 0x7f76c4071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c415aef0 con 0x7f76c4154b70 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 292428007 0 0) 0x7f76b8001670 con 0x7f76c4154b70 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 >> v1:172.21.15.62:6789/0 conn(0x7f76c4154b70 legacy=0x7f76c406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76ac005bc0 0x7f76ac005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76ac006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac006f50 0x7f76ac0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c406d3c0 con 0x7f76ac006f50 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4154f50 con 0x7f76ac005bc0 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac00dbd0 con 0x7f76c406dc80 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.110+0000 7f76c9e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac006f50 0x7f76ac0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c8a00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76ac006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9400640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76ac005bc0 0x7f76ac005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9e00640 1 -- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 msgr2=0x7f76ac006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c406dc80 0x7f76ac006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9e00640 1 -- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76ac005bc0 msgr2=0x7f76ac005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76ac005bc0 0x7f76ac005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9e00640 1 -- 172.21.15.62:0/857911227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac005730 con 0x7f76ac006f50 2024-03-28T18:00:24.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.111+0000 7f76c9400640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76ac005bc0 0x7f76ac005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:24.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.112+0000 7f76c9e00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac006f50 0x7f76ac0073d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009f90 tx=0x7f76ac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61518c3adc8e0be server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:24.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.112+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc039070 con 0x7f76ac006f50 2024-03-28T18:00:24.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.112+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc02cbd0 con 0x7f76ac006f50 2024-03-28T18:00:24.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.112+0000 7f76c3e00640 1 -- 172.21.15.62:0/857911227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc009c10 con 0x7f76ac006f50 2024-03-28T18:00:24.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 -- 172.21.15.62:0/857911227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac006f50 msgr2=0x7f76ac0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 --2- 172.21.15.62:0/857911227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac006f50 0x7f76ac0073d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009f90 tx=0x7f76ac007a40 comp rx=0 tx=0).stop 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76c9e00640 1 -- 172.21.15.62:0/857911227 reap_dead start 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 -- 172.21.15.62:0/857911227 shutdown_connections 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 -- 172.21.15.62:0/857911227 >> 172.21.15.62:0/857911227 conn(0x7f76c4066fb0 msgr2=0x7f76c40673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 -- 172.21.15.62:0/857911227 shutdown_connections 2024-03-28T18:00:24.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.113+0000 7f76cbe00640 1 -- 172.21.15.62:0/857911227 wait complete. 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.115+0000 7f76cbe00640 1 Processor -- start 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.115+0000 7f76cbe00640 1 -- start start 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c414f160 0x7f76c409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76c4153da0 0x7f76c40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415b790 con 0x7f76c40b3ba0 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415b4b0 con 0x7f76c4153da0 2024-03-28T18:00:24.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76cbe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40b2cb0 con 0x7f76c414f160 2024-03-28T18:00:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60310/0 (socket says 172.21.15.62:60310) 2024-03-28T18:00:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 -- 172.21.15.62:0/1921854159 learned_addr learned my addr 172.21.15.62:0/1921854159 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 -- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c414f160 msgr2=0x7f76c409aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c414f160 0x7f76c409aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 -- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76c4153da0 msgr2=0x7f76c40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76c4153da0 0x7f76c40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.116+0000 7f76c9e00640 1 -- 172.21.15.62:0/1921854159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c409b040 con 0x7f76c40b3ba0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76c9e00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f76bc037040 tx=0x7f76bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc039030 con 0x7f76c40b3ba0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc03a040 con 0x7f76c40b3ba0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c409b320 con 0x7f76c40b3ba0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc009c10 con 0x7f76c40b3ba0 2024-03-28T18:00:24.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.117+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c409b8e0 con 0x7f76c40b3ba0 2024-03-28T18:00:24.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.118+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c40680c0 con 0x7f76c40b3ba0 2024-03-28T18:00:24.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.119+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f76bc045030 con 0x7f76c40b3ba0 2024-03-28T18:00:24.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.120+0000 7f76c1600640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 0x7f76a406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:24.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.120+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76bc0ac690 con 0x7f76c40b3ba0 2024-03-28T18:00:24.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.121+0000 7f76c9400640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 0x7f76a406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:24.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.122+0000 7f76c9400640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 0x7f76a406e0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76b8001050 tx=0x7f76b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:24.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.130+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f76bc04c060 con 0x7f76c40b3ba0 2024-03-28T18:00:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.328+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76c4072c10 con 0x7f76a406bc00 2024-03-28T18:00:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.347+0000 7f76c1600640 1 -- 172.21.15.62:0/1921854159 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f76c4072c10 con 0x7f76a406bc00 2024-03-28T18:00:24.350 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:24.351 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 msgr2=0x7f76a406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 0x7f76a406e0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76b8001050 tx=0x7f76b8009040 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 msgr2=0x7f76c4151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f76bc037040 tx=0x7f76bc009750 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 shutdown_connections 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f76a406bc00 0x7f76a406e0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c40b3ba0 0x7f76c4151e10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76c4153da0 0x7f76c40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 --2- 172.21.15.62:0/1921854159 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f76c414f160 0x7f76c409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 >> 172.21.15.62:0/1921854159 conn(0x7f76c4066fb0 msgr2=0x7f76c406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:24.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 shutdown_connections 2024-03-28T18:00:24.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:24.351+0000 7f76cbe00640 1 -- 172.21.15.62:0/1921854159 wait complete. 2024-03-28T18:00:24.369 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-28T18:00:22.335925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3960832,"data_stored":2242797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001793"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569583+0000","last_change":"2024-03-28T18:00:16.569583+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569583+0000","last_undegraded":"2024-03-28T18:00:16.569583+0000","last_fullsized":"2024-03-28T18:00:16.569583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579654+0000","last_change":"2024-03-28T18:00:16.579654+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579654+0000","last_undegraded":"2024-03-28T18:00:16.579654+0000","last_fullsized":"2024-03-28T18:00:16.579654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561545+0000","last_change":"2024-03-28T18:00:16.561545+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561545+0000","last_undegraded":"2024-03-28T18:00:16.561545+0000","last_fullsized":"2024-03-28T18:00:16.561545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571390+0000","last_change":"2024-03-28T18:00:16.571390+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571390+0000","last_undegraded":"2024-03-28T18:00:16.571390+0000","last_fullsized":"2024-03-28T18:00:16.571390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557319+0000","last_change":"2024-03-28T18:00:16.557319+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557319+0000","last_undegraded":"2024-03-28T18:00:16.557319+0000","last_fullsized":"2024-03-28T18:00:16.557319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.580779+0000","last_change":"2024-03-28T18:00:16.580779+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.580779+0000","last_undegraded":"2024-03-28T18:00:16.580779+0000","last_fullsized":"2024-03-28T18:00:16.580779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568442+0000","last_change":"2024-03-28T18:00:16.568442+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568442+0000","last_undegraded":"2024-03-28T18:00:16.568442+0000","last_fullsized":"2024-03-28T18:00:16.568442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575897+0000","last_change":"2024-03-28T18:00:16.575897+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575897+0000","last_undegraded":"2024-03-28T18:00:16.575897+0000","last_fullsized":"2024-03-28T18:00:16.575897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576562+0000","last_change":"2024-03-28T18:00:16.576562+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576562+0000","last_undegraded":"2024-03-28T18:00:16.576562+0000","last_fullsized":"2024-03-28T18:00:16.576562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.568412+0000","last_change":"2024-03-28T18:00:16.568412+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.568412+0000","last_undegraded":"2024-03-28T18:00:16.568412+0000","last_fullsized":"2024-03-28T18:00:16.568412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569231+0000","last_change":"2024-03-28T18:00:16.569231+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569231+0000","last_undegraded":"2024-03-28T18:00:16.569231+0000","last_fullsized":"2024-03-28T18:00:16.569231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559639+0000","last_change":"2024-03-28T18:00:16.559639+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559639+0000","last_undegraded":"2024-03-28T18:00:16.559639+0000","last_fullsized":"2024-03-28T18:00:16.559639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556645+0000","last_change":"2024-03-28T18:00:16.556645+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556645+0000","last_undegraded":"2024-03-28T18:00:16.556645+0000","last_fullsized":"2024-03-28T18:00:16.556645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572147+0000","last_change":"2024-03-28T18:00:16.572147+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572147+0000","last_undegraded":"2024-03-28T18:00:16.572147+0000","last_fullsized":"2024-03-28T18:00:16.572147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557955+0000","last_change":"2024-03-28T18:00:16.557955+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557955+0000","last_undegraded":"2024-03-28T18:00:16.557955+0000","last_fullsized":"2024-03-28T18:00:16.557955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567619+0000","last_change":"2024-03-28T18:00:16.567619+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567619+0000","last_undegraded":"2024-03-28T18:00:16.567619+0000","last_fullsized":"2024-03-28T18:00:16.567619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567977+0000","last_change":"2024-03-28T18:00:16.567977+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567977+0000","last_undegraded":"2024-03-28T18:00:16.567977+0000","last_fullsized":"2024-03-28T18:00:16.567977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.577391+0000","last_change":"2024-03-28T18:00:16.577391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.577391+0000","last_undegraded":"2024-03-28T18:00:16.577391+0000","last_fullsized":"2024-03-28T18:00:16.577391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573104+0000","last_change":"2024-03-28T18:00:16.573104+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573104+0000","last_undegraded":"2024-03-28T18:00:16.573104+0000","last_fullsized":"2024-03-28T18:00:16.573104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555854+0000","last_change":"2024-03-28T18:00:16.555854+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555854+0000","last_undegraded":"2024-03-28T18:00:16.555854+0000","last_fullsized":"2024-03-28T18:00:16.555854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576380+0000","last_change":"2024-03-28T18:00:16.576380+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576380+0000","last_undegraded":"2024-03-28T18:00:16.576380+0000","last_fullsized":"2024-03-28T18:00:16.576380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571362+0000","last_change":"2024-03-28T18:00:16.571362+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571362+0000","last_undegraded":"2024-03-28T18:00:16.571362+0000","last_fullsized":"2024-03-28T18:00:16.571362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561762+0000","last_change":"2024-03-28T18:00:16.561762+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561762+0000","last_undegraded":"2024-03-28T18:00:16.561762+0000","last_fullsized":"2024-03-28T18:00:16.561762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558028+0000","last_change":"2024-03-28T18:00:16.558028+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558028+0000","last_undegraded":"2024-03-28T18:00:16.558028+0000","last_fullsized":"2024-03-28T18:00:16.558028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569204+0000","last_change":"2024-03-28T18:00:16.569204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569204+0000","last_undegraded":"2024-03-28T18:00:16.569204+0000","last_fullsized":"2024-03-28T18:00:16.569204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576919+0000","last_change":"2024-03-28T18:00:16.576919+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576919+0000","last_undegraded":"2024-03-28T18:00:16.576919+0000","last_fullsized":"2024-03-28T18:00:16.576919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566641+0000","last_change":"2024-03-28T18:00:16.566641+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566641+0000","last_undegraded":"2024-03-28T18:00:16.566641+0000","last_fullsized":"2024-03-28T18:00:16.566641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.558269+0000","last_change":"2024-03-28T18:00:16.558269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.558269+0000","last_undegraded":"2024-03-28T18:00:16.558269+0000","last_fullsized":"2024-03-28T18:00:16.558269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575581+0000","last_change":"2024-03-28T18:00:16.575581+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575581+0000","last_undegraded":"2024-03-28T18:00:16.575581+0000","last_fullsized":"2024-03-28T18:00:16.575581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.572204+0000","last_change":"2024-03-28T18:00:16.572204+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.572204+0000","last_undegraded":"2024-03-28T18:00:16.572204+0000","last_fullsized":"2024-03-28T18:00:16.572204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571667+0000","last_change":"2024-03-28T18:00:16.571667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571667+0000","last_undegraded":"2024-03-28T18:00:16.571667+0000","last_fullsized":"2024-03-28T18:00:16.571667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574076+0000","last_change":"2024-03-28T18:00:16.574076+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574076+0000","last_undegraded":"2024-03-28T18:00:16.574076+0000","last_fullsized":"2024-03-28T18:00:16.574076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562017+0000","last_change":"2024-03-28T18:00:16.562017+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562017+0000","last_undegraded":"2024-03-28T18:00:16.562017+0000","last_fullsized":"2024-03-28T18:00:16.562017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588360+0000","last_change":"2024-03-28T18:00:16.586041+0000","last_active":"2024-03-28T18:00:16.588360+0000","last_peered":"2024-03-28T18:00:16.588360+0000","last_clean":"2024-03-28T18:00:16.588360+0000","last_became_active":"2024-03-28T18:00:16.585654+0000","last_became_peered":"2024-03-28T18:00:16.585654+0000","last_unstale":"2024-03-28T18:00:16.588360+0000","last_undegraded":"2024-03-28T18:00:16.588360+0000","last_fullsized":"2024-03-28T18:00:16.588360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567054+0000","last_change":"2024-03-28T18:00:16.567054+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567054+0000","last_undegraded":"2024-03-28T18:00:16.567054+0000","last_fullsized":"2024-03-28T18:00:16.567054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586926+0000","last_change":"2024-03-28T18:00:16.582651+0000","last_active":"2024-03-28T18:00:16.586926+0000","last_peered":"2024-03-28T18:00:16.586926+0000","last_clean":"2024-03-28T18:00:16.586926+0000","last_became_active":"2024-03-28T18:00:16.582248+0000","last_became_peered":"2024-03-28T18:00:16.582248+0000","last_unstale":"2024-03-28T18:00:16.586926+0000","last_undegraded":"2024-03-28T18:00:16.586926+0000","last_fullsized":"2024-03-28T18:00:16.586926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571368+0000","last_change":"2024-03-28T18:00:16.571368+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571368+0000","last_undegraded":"2024-03-28T18:00:16.571368+0000","last_fullsized":"2024-03-28T18:00:16.571368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592321+0000","last_change":"2024-03-28T18:00:16.588264+0000","last_active":"2024-03-28T18:00:16.592321+0000","last_peered":"2024-03-28T18:00:16.592321+0000","last_clean":"2024-03-28T18:00:16.592321+0000","last_became_active":"2024-03-28T18:00:16.587925+0000","last_became_peered":"2024-03-28T18:00:16.587925+0000","last_unstale":"2024-03-28T18:00:16.592321+0000","last_undegraded":"2024-03-28T18:00:16.592321+0000","last_fullsized":"2024-03-28T18:00:16.592321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565721+0000","last_change":"2024-03-28T18:00:16.565721+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565721+0000","last_undegraded":"2024-03-28T18:00:16.565721+0000","last_fullsized":"2024-03-28T18:00:16.565721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579273+0000","last_change":"2024-03-28T18:00:16.577746+0000","last_active":"2024-03-28T18:00:16.579273+0000","last_peered":"2024-03-28T18:00:16.579273+0000","last_clean":"2024-03-28T18:00:16.579273+0000","last_became_active":"2024-03-28T18:00:16.577340+0000","last_became_peered":"2024-03-28T18:00:16.577340+0000","last_unstale":"2024-03-28T18:00:16.579273+0000","last_undegraded":"2024-03-28T18:00:16.579273+0000","last_fullsized":"2024-03-28T18:00:16.579273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570787+0000","last_change":"2024-03-28T18:00:16.570787+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570787+0000","last_undegraded":"2024-03-28T18:00:16.570787+0000","last_fullsized":"2024-03-28T18:00:16.570787+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593331+0000","last_change":"2024-03-28T18:00:16.591523+0000","last_active":"2024-03-28T18:00:16.593331+0000","last_peered":"2024-03-28T18:00:16.593331+0000","last_clean":"2024-03-28T18:00:16.593331+0000","last_became_active":"2024-03-28T18:00:16.591203+0000","last_became_peered":"2024-03-28T18:00:16.591203+0000","last_unstale":"2024-03-28T18:00:16.593331+0000","last_undegraded":"2024-03-28T18:00:16.593331+0000","last_fullsized":"2024-03-28T18:00:16.593331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:13.152406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.559660+0000","last_change":"2024-03-28T18:00:16.559660+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.559660+0000","last_undegraded":"2024-03-28T18:00:16.559660+0000","last_fullsized":"2024-03-28T18:00:16.559660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.593355+0000","last_change":"2024-03-28T18:00:16.591608+0000","last_active":"2024-03-28T18:00:16.593355+0000","last_peered":"2024-03-28T18:00:16.593355+0000","last_clean":"2024-03-28T18:00:16.593355+0000","last_became_active":"2024-03-28T18:00:16.591289+0000","last_became_peered":"2024-03-28T18:00:16.591289+0000","last_unstale":"2024-03-28T18:00:16.593355+0000","last_undegraded":"2024-03-28T18:00:16.593355+0000","last_fullsized":"2024-03-28T18:00:16.593355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:23.345799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574763+0000","last_change":"2024-03-28T18:00:16.574763+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574763+0000","last_undegraded":"2024-03-28T18:00:16.574763+0000","last_fullsized":"2024-03-28T18:00:16.574763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589450+0000","last_change":"2024-03-28T18:00:16.581766+0000","last_active":"2024-03-28T18:00:16.589450+0000","last_peered":"2024-03-28T18:00:16.589450+0000","last_clean":"2024-03-28T18:00:16.589450+0000","last_became_active":"2024-03-28T18:00:16.581506+0000","last_became_peered":"2024-03-28T18:00:16.581506+0000","last_unstale":"2024-03-28T18:00:16.589450+0000","last_undegraded":"2024-03-28T18:00:16.589450+0000","last_fullsized":"2024-03-28T18:00:16.589450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.564269+0000","last_change":"2024-03-28T18:00:16.564269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.564269+0000","last_undegraded":"2024-03-28T18:00:16.564269+0000","last_fullsized":"2024-03-28T18:00:16.564269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590746+0000","last_change":"2024-03-28T18:00:16.589309+0000","last_active":"2024-03-28T18:00:16.590746+0000","last_peered":"2024-03-28T18:00:16.590746+0000","last_clean":"2024-03-28T18:00:16.590746+0000","last_became_active":"2024-03-28T18:00:16.588995+0000","last_became_peered":"2024-03-28T18:00:16.588995+0000","last_unstale":"2024-03-28T18:00:16.590746+0000","last_undegraded":"2024-03-28T18:00:16.590746+0000","last_fullsized":"2024-03-28T18:00:16.590746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:10.952085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560002+0000","last_change":"2024-03-28T18:00:16.560002+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560002+0000","last_undegraded":"2024-03-28T18:00:16.560002+0000","last_fullsized":"2024-03-28T18:00:16.560002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592334+0000","last_change":"2024-03-28T18:00:16.588159+0000","last_active":"2024-03-28T18:00:16.592334+0000","last_peered":"2024-03-28T18:00:16.592334+0000","last_clean":"2024-03-28T18:00:16.592334+0000","last_became_active":"2024-03-28T18:00:16.587891+0000","last_became_peered":"2024-03-28T18:00:16.587891+0000","last_unstale":"2024-03-28T18:00:16.592334+0000","last_undegraded":"2024-03-28T18:00:16.592334+0000","last_fullsized":"2024-03-28T18:00:16.592334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.574440+0000","last_change":"2024-03-28T18:00:16.574440+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.574440+0000","last_undegraded":"2024-03-28T18:00:16.574440+0000","last_fullsized":"2024-03-28T18:00:16.574440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592843+0000","last_change":"2024-03-28T18:00:16.590209+0000","last_active":"2024-03-28T18:00:16.592843+0000","last_peered":"2024-03-28T18:00:16.592843+0000","last_clean":"2024-03-28T18:00:16.592843+0000","last_became_active":"2024-03-28T18:00:16.589874+0000","last_became_peered":"2024-03-28T18:00:16.589874+0000","last_unstale":"2024-03-28T18:00:16.592843+0000","last_undegraded":"2024-03-28T18:00:16.592843+0000","last_fullsized":"2024-03-28T18:00:16.592843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.573906+0000","last_change":"2024-03-28T18:00:16.573906+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.573906+0000","last_undegraded":"2024-03-28T18:00:16.573906+0000","last_fullsized":"2024-03-28T18:00:16.573906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588152+0000","last_change":"2024-03-28T18:00:16.583925+0000","last_active":"2024-03-28T18:00:16.588152+0000","last_peered":"2024-03-28T18:00:16.588152+0000","last_clean":"2024-03-28T18:00:16.588152+0000","last_became_active":"2024-03-28T18:00:16.583635+0000","last_became_peered":"2024-03-28T18:00:16.583635+0000","last_unstale":"2024-03-28T18:00:16.588152+0000","last_undegraded":"2024-03-28T18:00:16.588152+0000","last_fullsized":"2024-03-28T18:00:16.588152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562694+0000","last_change":"2024-03-28T18:00:16.562694+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562694+0000","last_undegraded":"2024-03-28T18:00:16.562694+0000","last_fullsized":"2024-03-28T18:00:16.562694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.597940+0000","last_change":"2024-03-28T18:00:16.593104+0000","last_active":"2024-03-28T18:00:16.597940+0000","last_peered":"2024-03-28T18:00:16.597940+0000","last_clean":"2024-03-28T18:00:16.597940+0000","last_became_active":"2024-03-28T18:00:16.592771+0000","last_became_peered":"2024-03-28T18:00:16.592771+0000","last_unstale":"2024-03-28T18:00:16.597940+0000","last_undegraded":"2024-03-28T18:00:16.597940+0000","last_fullsized":"2024-03-28T18:00:16.597940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.565690+0000","last_change":"2024-03-28T18:00:16.565690+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.565690+0000","last_undegraded":"2024-03-28T18:00:16.565690+0000","last_fullsized":"2024-03-28T18:00:16.565690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589009+0000","last_change":"2024-03-28T18:00:16.584849+0000","last_active":"2024-03-28T18:00:16.589009+0000","last_peered":"2024-03-28T18:00:16.589009+0000","last_clean":"2024-03-28T18:00:16.589009+0000","last_became_active":"2024-03-28T18:00:16.584543+0000","last_became_peered":"2024-03-28T18:00:16.584543+0000","last_unstale":"2024-03-28T18:00:16.589009+0000","last_undegraded":"2024-03-28T18:00:16.589009+0000","last_fullsized":"2024-03-28T18:00:16.589009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:28:00.623248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.579339+0000","last_change":"2024-03-28T18:00:16.579339+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.579339+0000","last_undegraded":"2024-03-28T18:00:16.579339+0000","last_fullsized":"2024-03-28T18:00:16.579339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583382+0000","last_change":"2024-03-28T18:00:16.580448+0000","last_active":"2024-03-28T18:00:16.583382+0000","last_peered":"2024-03-28T18:00:16.583382+0000","last_clean":"2024-03-28T18:00:16.583382+0000","last_became_active":"2024-03-28T18:00:16.579745+0000","last_became_peered":"2024-03-28T18:00:16.579745+0000","last_unstale":"2024-03-28T18:00:16.583382+0000","last_undegraded":"2024-03-28T18:00:16.583382+0000","last_fullsized":"2024-03-28T18:00:16.583382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571904+0000","last_change":"2024-03-28T18:00:16.571904+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571904+0000","last_undegraded":"2024-03-28T18:00:16.571904+0000","last_fullsized":"2024-03-28T18:00:16.571904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592117+0000","last_change":"2024-03-28T18:00:16.587497+0000","last_active":"2024-03-28T18:00:16.592117+0000","last_peered":"2024-03-28T18:00:16.592117+0000","last_clean":"2024-03-28T18:00:16.592117+0000","last_became_active":"2024-03-28T18:00:16.586180+0000","last_became_peered":"2024-03-28T18:00:16.586180+0000","last_unstale":"2024-03-28T18:00:16.592117+0000","last_undegraded":"2024-03-28T18:00:16.592117+0000","last_fullsized":"2024-03-28T18:00:16.592117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.566667+0000","last_change":"2024-03-28T18:00:16.566667+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.566667+0000","last_undegraded":"2024-03-28T18:00:16.566667+0000","last_fullsized":"2024-03-28T18:00:16.566667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589771+0000","last_change":"2024-03-28T18:00:16.581297+0000","last_active":"2024-03-28T18:00:16.589771+0000","last_peered":"2024-03-28T18:00:16.589771+0000","last_clean":"2024-03-28T18:00:16.589771+0000","last_became_active":"2024-03-28T18:00:16.580937+0000","last_became_peered":"2024-03-28T18:00:16.580937+0000","last_unstale":"2024-03-28T18:00:16.589771+0000","last_undegraded":"2024-03-28T18:00:16.589771+0000","last_fullsized":"2024-03-28T18:00:16.589771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.571676+0000","last_change":"2024-03-28T18:00:16.571676+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.571676+0000","last_undegraded":"2024-03-28T18:00:16.571676+0000","last_fullsized":"2024-03-28T18:00:16.571676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592286+0000","last_change":"2024-03-28T18:00:16.589262+0000","last_active":"2024-03-28T18:00:16.592286+0000","last_peered":"2024-03-28T18:00:16.592286+0000","last_clean":"2024-03-28T18:00:16.592286+0000","last_became_active":"2024-03-28T18:00:16.588786+0000","last_became_peered":"2024-03-28T18:00:16.588786+0000","last_unstale":"2024-03-28T18:00:16.592286+0000","last_undegraded":"2024-03-28T18:00:16.592286+0000","last_fullsized":"2024-03-28T18:00:16.592286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561624+0000","last_change":"2024-03-28T18:00:16.561624+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561624+0000","last_undegraded":"2024-03-28T18:00:16.561624+0000","last_fullsized":"2024-03-28T18:00:16.561624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.589122+0000","last_change":"2024-03-28T18:00:16.587380+0000","last_active":"2024-03-28T18:00:16.589122+0000","last_peered":"2024-03-28T18:00:16.589122+0000","last_clean":"2024-03-28T18:00:16.589122+0000","last_became_active":"2024-03-28T18:00:16.586842+0000","last_became_peered":"2024-03-28T18:00:16.586842+0000","last_unstale":"2024-03-28T18:00:16.589122+0000","last_undegraded":"2024-03-28T18:00:16.589122+0000","last_fullsized":"2024-03-28T18:00:16.589122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.737137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.570537+0000","last_change":"2024-03-28T18:00:16.570537+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.570537+0000","last_undegraded":"2024-03-28T18:00:16.570537+0000","last_fullsized":"2024-03-28T18:00:16.570537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588036+0000","last_change":"2024-03-28T18:00:16.581961+0000","last_active":"2024-03-28T18:00:16.588036+0000","last_peered":"2024-03-28T18:00:16.588036+0000","last_clean":"2024-03-28T18:00:16.588036+0000","last_became_active":"2024-03-28T18:00:16.581694+0000","last_became_peered":"2024-03-28T18:00:16.581694+0000","last_unstale":"2024-03-28T18:00:16.588036+0000","last_undegraded":"2024-03-28T18:00:16.588036+0000","last_fullsized":"2024-03-28T18:00:16.588036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:58:38.567836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556811+0000","last_change":"2024-03-28T18:00:16.556811+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556811+0000","last_undegraded":"2024-03-28T18:00:16.556811+0000","last_fullsized":"2024-03-28T18:00:16.556811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.591372+0000","last_change":"2024-03-28T18:00:16.590452+0000","last_active":"2024-03-28T18:00:16.591372+0000","last_peered":"2024-03-28T18:00:16.591372+0000","last_clean":"2024-03-28T18:00:16.591372+0000","last_became_active":"2024-03-28T18:00:16.590148+0000","last_became_peered":"2024-03-28T18:00:16.590148+0000","last_unstale":"2024-03-28T18:00:16.591372+0000","last_undegraded":"2024-03-28T18:00:16.591372+0000","last_fullsized":"2024-03-28T18:00:16.591372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:59.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555930+0000","last_change":"2024-03-28T18:00:16.555930+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555930+0000","last_undegraded":"2024-03-28T18:00:16.555930+0000","last_fullsized":"2024-03-28T18:00:16.555930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.578677+0000","last_change":"2024-03-28T18:00:16.574301+0000","last_active":"2024-03-28T18:00:16.578677+0000","last_peered":"2024-03-28T18:00:16.578677+0000","last_clean":"2024-03-28T18:00:16.578677+0000","last_became_active":"2024-03-28T18:00:16.573671+0000","last_became_peered":"2024-03-28T18:00:16.573671+0000","last_unstale":"2024-03-28T18:00:16.578677+0000","last_undegraded":"2024-03-28T18:00:16.578677+0000","last_fullsized":"2024-03-28T18:00:16.578677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.555199+0000","last_change":"2024-03-28T18:00:16.555199+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.555199+0000","last_undegraded":"2024-03-28T18:00:16.555199+0000","last_fullsized":"2024-03-28T18:00:16.555199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.586129+0000","last_change":"2024-03-28T18:00:16.584421+0000","last_active":"2024-03-28T18:00:16.586129+0000","last_peered":"2024-03-28T18:00:16.586129+0000","last_clean":"2024-03-28T18:00:16.586129+0000","last_became_active":"2024-03-28T18:00:16.584113+0000","last_became_peered":"2024-03-28T18:00:16.584113+0000","last_unstale":"2024-03-28T18:00:16.586129+0000","last_undegraded":"2024-03-28T18:00:16.586129+0000","last_fullsized":"2024-03-28T18:00:16.586129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.556334+0000","last_change":"2024-03-28T18:00:16.556334+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.556334+0000","last_undegraded":"2024-03-28T18:00:16.556334+0000","last_fullsized":"2024-03-28T18:00:16.556334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.582176+0000","last_change":"2024-03-28T18:00:16.578260+0000","last_active":"2024-03-28T18:00:16.582176+0000","last_peered":"2024-03-28T18:00:16.582176+0000","last_clean":"2024-03-28T18:00:16.582176+0000","last_became_active":"2024-03-28T18:00:16.577824+0000","last_became_peered":"2024-03-28T18:00:16.577824+0000","last_unstale":"2024-03-28T18:00:16.582176+0000","last_undegraded":"2024-03-28T18:00:16.582176+0000","last_fullsized":"2024-03-28T18:00:16.582176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:13:08.234116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569631+0000","last_change":"2024-03-28T18:00:16.569631+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569631+0000","last_undegraded":"2024-03-28T18:00:16.569631+0000","last_fullsized":"2024-03-28T18:00:16.569631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.588830+0000","last_change":"2024-03-28T18:00:16.584736+0000","last_active":"2024-03-28T18:00:16.588830+0000","last_peered":"2024-03-28T18:00:16.588830+0000","last_clean":"2024-03-28T18:00:16.588830+0000","last_became_active":"2024-03-28T18:00:16.584455+0000","last_became_peered":"2024-03-28T18:00:16.584455+0000","last_unstale":"2024-03-28T18:00:16.588830+0000","last_undegraded":"2024-03-28T18:00:16.588830+0000","last_fullsized":"2024-03-28T18:00:16.588830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.557269+0000","last_change":"2024-03-28T18:00:16.557269+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.557269+0000","last_undegraded":"2024-03-28T18:00:16.557269+0000","last_fullsized":"2024-03-28T18:00:16.557269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.584256+0000","last_change":"2024-03-28T18:00:16.583053+0000","last_active":"2024-03-28T18:00:16.584256+0000","last_peered":"2024-03-28T18:00:16.584256+0000","last_clean":"2024-03-28T18:00:16.584256+0000","last_became_active":"2024-03-28T18:00:16.582775+0000","last_became_peered":"2024-03-28T18:00:16.582775+0000","last_unstale":"2024-03-28T18:00:16.584256+0000","last_undegraded":"2024-03-28T18:00:16.584256+0000","last_fullsized":"2024-03-28T18:00:16.584256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.560345+0000","last_change":"2024-03-28T18:00:16.560345+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.560345+0000","last_undegraded":"2024-03-28T18:00:16.560345+0000","last_fullsized":"2024-03-28T18:00:16.560345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579729+0000","last_change":"2024-03-28T18:00:16.575792+0000","last_active":"2024-03-28T18:00:16.579729+0000","last_peered":"2024-03-28T18:00:16.579729+0000","last_clean":"2024-03-28T18:00:16.579729+0000","last_became_active":"2024-03-28T18:00:16.575475+0000","last_became_peered":"2024-03-28T18:00:16.575475+0000","last_unstale":"2024-03-28T18:00:16.579729+0000","last_undegraded":"2024-03-28T18:00:16.579729+0000","last_fullsized":"2024-03-28T18:00:16.579729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.562722+0000","last_change":"2024-03-28T18:00:16.562722+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.562722+0000","last_undegraded":"2024-03-28T18:00:16.562722+0000","last_fullsized":"2024-03-28T18:00:16.562722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579125+0000","last_change":"2024-03-28T18:00:16.576446+0000","last_active":"2024-03-28T18:00:16.579125+0000","last_peered":"2024-03-28T18:00:16.579125+0000","last_clean":"2024-03-28T18:00:16.579125+0000","last_became_active":"2024-03-28T18:00:16.575444+0000","last_became_peered":"2024-03-28T18:00:16.575444+0000","last_unstale":"2024-03-28T18:00:16.579125+0000","last_undegraded":"2024-03-28T18:00:16.579125+0000","last_fullsized":"2024-03-28T18:00:16.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.567784+0000","last_change":"2024-03-28T18:00:16.567784+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.567784+0000","last_undegraded":"2024-03-28T18:00:16.567784+0000","last_fullsized":"2024-03-28T18:00:16.567784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.577769+0000","last_change":"2024-03-28T18:00:16.573496+0000","last_active":"2024-03-28T18:00:16.577769+0000","last_peered":"2024-03-28T18:00:16.577769+0000","last_clean":"2024-03-28T18:00:16.577769+0000","last_became_active":"2024-03-28T18:00:16.573041+0000","last_became_peered":"2024-03-28T18:00:16.573041+0000","last_unstale":"2024-03-28T18:00:16.577769+0000","last_undegraded":"2024-03-28T18:00:16.577769+0000","last_fullsized":"2024-03-28T18:00:16.577769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:56.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.561746+0000","last_change":"2024-03-28T18:00:16.561746+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.561746+0000","last_undegraded":"2024-03-28T18:00:16.561746+0000","last_fullsized":"2024-03-28T18:00:16.561746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.583640+0000","last_change":"2024-03-28T18:00:16.581384+0000","last_active":"2024-03-28T18:00:16.583640+0000","last_peered":"2024-03-28T18:00:16.583640+0000","last_clean":"2024-03-28T18:00:16.583640+0000","last_became_active":"2024-03-28T18:00:16.581030+0000","last_became_peered":"2024-03-28T18:00:16.581030+0000","last_unstale":"2024-03-28T18:00:16.583640+0000","last_undegraded":"2024-03-28T18:00:16.583640+0000","last_fullsized":"2024-03-28T18:00:16.583640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.569391+0000","last_change":"2024-03-28T18:00:16.569391+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.569391+0000","last_undegraded":"2024-03-28T18:00:16.569391+0000","last_fullsized":"2024-03-28T18:00:16.569391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.587915+0000","last_change":"2024-03-28T18:00:16.582503+0000","last_active":"2024-03-28T18:00:16.587915+0000","last_peered":"2024-03-28T18:00:16.587915+0000","last_clean":"2024-03-28T18:00:16.587915+0000","last_became_active":"2024-03-28T18:00:16.581354+0000","last_became_peered":"2024-03-28T18:00:16.581354+0000","last_unstale":"2024-03-28T18:00:16.587915+0000","last_undegraded":"2024-03-28T18:00:16.587915+0000","last_fullsized":"2024-03-28T18:00:16.587915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:02.785011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.575167+0000","last_change":"2024-03-28T18:00:16.575167+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.575167+0000","last_undegraded":"2024-03-28T18:00:16.575167+0000","last_fullsized":"2024-03-28T18:00:16.575167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.592189+0000","last_change":"2024-03-28T18:00:16.587890+0000","last_active":"2024-03-28T18:00:16.592189+0000","last_peered":"2024-03-28T18:00:16.592189+0000","last_clean":"2024-03-28T18:00:16.592189+0000","last_became_active":"2024-03-28T18:00:16.587564+0000","last_became_peered":"2024-03-28T18:00:16.587564+0000","last_unstale":"2024-03-28T18:00:16.592189+0000","last_undegraded":"2024-03-28T18:00:16.592189+0000","last_fullsized":"2024-03-28T18:00:16.592189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-28T18:00:16.576464+0000","last_change":"2024-03-28T18:00:16.576464+0000","last_active":"2024-03-28T18:00:16.533783+0000","last_peered":"2024-03-28T18:00:16.533783+0000","last_clean":"2024-03-28T18:00:16.533783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T18:00:16.576464+0000","last_undegraded":"2024-03-28T18:00:16.576464+0000","last_fullsized":"2024-03-28T18:00:16.576464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.590990+0000","last_change":"2024-03-28T18:00:16.589804+0000","last_active":"2024-03-28T18:00:16.590990+0000","last_peered":"2024-03-28T18:00:16.590990+0000","last_clean":"2024-03-28T18:00:16.590990+0000","last_became_active":"2024-03-28T18:00:16.589477+0000","last_became_peered":"2024-03-28T18:00:16.589477+0000","last_unstale":"2024-03-28T18:00:16.590990+0000","last_undegraded":"2024-03-28T18:00:16.590990+0000","last_fullsized":"2024-03-28T18:00:16.590990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T18:00:16.579800+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T18:00:16.579800+0000","last_peered":"2024-03-28T18:00:16.579800+0000","last_clean":"2024-03-28T18:00:16.579800+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T18:00:16.579800+0000","last_undegraded":"2024-03-28T18:00:16.579800+0000","last_fullsized":"2024-03-28T18:00:16.579800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:00:27.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:00:27.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.592+0000 7fd900800640 1 Processor -- start 2024-03-28T18:00:27.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.592+0000 7fd900800640 1 -- start start 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd900800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8f80b3310 0x7fd8f80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd900800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80d24a0 con 0x7fd8f8071930 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd900800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80d2720 con 0x7fd8f8071d10 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd900800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80d3330 con 0x7fd8f80b3310 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd8fde00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd8f8071930 0x7fd8f80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41470/0 (socket says 172.21.15.62:41470) 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd8fde00640 1 -- 172.21.15.62:0/2948692085 learned_addr learned my addr 172.21.15.62:0/2948692085 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.595+0000 7fd8fd400640 1 --2- 172.21.15.62:0/2948692085 >> v2:172.21.15.62:3300/0 conn(0x7fd8f80b3310 0x7fd8f80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.596+0000 7fd8fca00640 1 -- 172.21.15.62:0/2948692085 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734155955 0 0) 0x7fd8ec001110 con 0x7fd8f8071930 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.596+0000 7fd8fca00640 1 -- 172.21.15.62:0/2948692085 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e0003520 con 0x7fd8f8071930 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.596+0000 7fd8fd400640 1 -- 172.21.15.62:0/2948692085 >> v1:172.21.15.81:6789/0 conn(0x7fd8f8071d10 legacy=0x7fd8f80b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.596+0000 7fd8fd400640 1 -- 172.21.15.62:0/2948692085 >> v1:172.21.15.62:6789/0 conn(0x7fd8f8071930 legacy=0x7fd8f80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.596+0000 7fd8fd400640 1 -- 172.21.15.62:0/2948692085 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80d2a00 con 0x7fd8f80b3310 2024-03-28T18:00:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.598+0000 7fd8fd400640 1 --2- 172.21.15.62:0/2948692085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f80b3310 0x7fd8f80ceef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80097a0 tx=0x7fd8e802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eac7ca481a58c59 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.599+0000 7fd8fca00640 1 -- 172.21.15.62:0/2948692085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e802f810 con 0x7fd8f80b3310 2024-03-28T18:00:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.599+0000 7fd8fca00640 1 -- 172.21.15.62:0/2948692085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e802f9b0 con 0x7fd8f80b3310 2024-03-28T18:00:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.599+0000 7fd8fca00640 1 -- 172.21.15.62:0/2948692085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e802fd80 con 0x7fd8f80b3310 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.599+0000 7fd900800640 1 -- 172.21.15.62:0/2948692085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f80b3310 msgr2=0x7fd8f80ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.599+0000 7fd900800640 1 --2- 172.21.15.62:0/2948692085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f80b3310 0x7fd8f80ceef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80097a0 tx=0x7fd8e802b9f0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.600+0000 7fd900800640 1 -- 172.21.15.62:0/2948692085 shutdown_connections 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.600+0000 7fd900800640 1 --2- 172.21.15.62:0/2948692085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f80b3310 0x7fd8f80ceef0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.600+0000 7fd900800640 1 -- 172.21.15.62:0/2948692085 >> 172.21.15.62:0/2948692085 conn(0x7fd8f80d3ef0 msgr2=0x7fd8f80d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.600+0000 7fd900800640 1 -- 172.21.15.62:0/2948692085 shutdown_connections 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.600+0000 7fd900800640 1 -- 172.21.15.62:0/2948692085 wait complete. 2024-03-28T18:00:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.601+0000 7fd900800640 1 Processor -- start 2024-03-28T18:00:27.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 -- start start 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8f8071930 0x7fd8f815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8f80b3310 0x7fd8f80b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80d4dd0 con 0x7fd8f8071d10 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80d2fc0 con 0x7fd8f8071930 2024-03-28T18:00:27.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.602+0000 7fd900800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80d37c0 con 0x7fd8f80b3310 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60332/0 (socket says 172.21.15.62:60332) 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 -- 172.21.15.62:0/3593928438 learned_addr learned my addr 172.21.15.62:0/3593928438 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fde00640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8f8071930 0x7fd8f815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 -- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8f80b3310 msgr2=0x7fd8f80b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fe800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8f80b3310 0x7fd8f80b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8f80b3310 0x7fd8f80b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 -- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8f8071930 msgr2=0x7fd8f815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8f8071930 0x7fd8f815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.603+0000 7fd8fd400640 1 -- 172.21.15.62:0/3593928438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f815ecb0 con 0x7fd8f8071d10 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd8fd400640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e802c000 tx=0x7fd8e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8046070 con 0x7fd8f8071d10 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8037810 con 0x7fd8f8071d10 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8037c00 con 0x7fd8f8071d10 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f815ef50 con 0x7fd8f8071d10 2024-03-28T18:00:27.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.604+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f815f510 con 0x7fd8f8071d10 2024-03-28T18:00:27.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.606+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f80680c0 con 0x7fd8f8071d10 2024-03-28T18:00:27.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.606+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd8e8049070 con 0x7fd8f8071d10 2024-03-28T18:00:27.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.608+0000 7fd8f2000640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 0x7fd8dc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:27.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.608+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8e80ac520 con 0x7fd8f8071d10 2024-03-28T18:00:27.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.608+0000 7fd8fde00640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 0x7fd8dc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:27.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.610+0000 7fd8fde00640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 0x7fd8dc06e090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec007fd0 tx=0x7fd8ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:27.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.612+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fd8e8033030 con 0x7fd8f8071d10 2024-03-28T18:00:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.844+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 --> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8f81e9060 con 0x7fd8dc06bbd0 2024-03-28T18:00:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.862+0000 7fd8f2000640 1 -- 172.21.15.62:0/3593928438 <== mgr.4106 v2:172.21.15.81:6832/1316232289 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7fd8f81e9060 con 0x7fd8dc06bbd0 2024-03-28T18:00:27.866 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 msgr2=0x7fd8dc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 0x7fd8dc06e090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec007fd0 tx=0x7fd8ec009040 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 msgr2=0x7fd8f80b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e802c000 tx=0x7fd8e8004730 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 shutdown_connections 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fd8dc06bbd0 0x7fd8dc06e090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fd8f80b3310 0x7fd8f80b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f8071d10 0x7fd8f80b8230 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 --2- 172.21.15.62:0/3593928438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8f8071930 0x7fd8f815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 >> 172.21.15.62:0/3593928438 conn(0x7fd8f80d3ef0 msgr2=0x7fd8f8161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 shutdown_connections 2024-03-28T18:00:27.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:27.868+0000 7fd900800640 1 -- 172.21.15.62:0/3593928438 wait complete. 2024-03-28T18:00:27.889 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-03-28T18:00:26.347917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220320,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754400,"statfs":{"total":773094113280,"available":772868505600,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.790951"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182106+0000","last_change":"2024-03-28T18:00:17.589509+0000","last_active":"2024-03-28T18:00:19.182106+0000","last_peered":"2024-03-28T18:00:19.182106+0000","last_clean":"2024-03-28T18:00:19.182106+0000","last_became_active":"2024-03-28T18:00:17.589250+0000","last_became_peered":"2024-03-28T18:00:17.589250+0000","last_unstale":"2024-03-28T18:00:19.182106+0000","last_undegraded":"2024-03-28T18:00:19.182106+0000","last_fullsized":"2024-03-28T18:00:19.182106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:28.373593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183092+0000","last_change":"2024-03-28T18:00:17.570530+0000","last_active":"2024-03-28T18:00:19.183092+0000","last_peered":"2024-03-28T18:00:19.183092+0000","last_clean":"2024-03-28T18:00:19.183092+0000","last_became_active":"2024-03-28T18:00:17.566371+0000","last_became_peered":"2024-03-28T18:00:17.566371+0000","last_unstale":"2024-03-28T18:00:19.183092+0000","last_undegraded":"2024-03-28T18:00:19.183092+0000","last_fullsized":"2024-03-28T18:00:19.183092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:36:32.575303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.563529+0000","last_change":"2024-03-28T18:00:17.580072+0000","last_active":"2024-03-28T18:00:18.563529+0000","last_peered":"2024-03-28T18:00:18.563529+0000","last_clean":"2024-03-28T18:00:18.563529+0000","last_became_active":"2024-03-28T18:00:17.579413+0000","last_became_peered":"2024-03-28T18:00:17.579413+0000","last_unstale":"2024-03-28T18:00:18.563529+0000","last_undegraded":"2024-03-28T18:00:18.563529+0000","last_fullsized":"2024-03-28T18:00:18.563529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:12:03.104842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.179888+0000","last_change":"2024-03-28T18:00:17.598805+0000","last_active":"2024-03-28T18:00:19.179888+0000","last_peered":"2024-03-28T18:00:19.179888+0000","last_clean":"2024-03-28T18:00:19.179888+0000","last_became_active":"2024-03-28T18:00:17.598198+0000","last_became_peered":"2024-03-28T18:00:17.598198+0000","last_unstale":"2024-03-28T18:00:19.179888+0000","last_undegraded":"2024-03-28T18:00:19.179888+0000","last_fullsized":"2024-03-28T18:00:19.179888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:15:06.195750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.560826+0000","last_change":"2024-03-28T18:00:17.579956+0000","last_active":"2024-03-28T18:00:18.560826+0000","last_peered":"2024-03-28T18:00:18.560826+0000","last_clean":"2024-03-28T18:00:18.560826+0000","last_became_active":"2024-03-28T18:00:17.579253+0000","last_became_peered":"2024-03-28T18:00:17.579253+0000","last_unstale":"2024-03-28T18:00:18.560826+0000","last_undegraded":"2024-03-28T18:00:18.560826+0000","last_fullsized":"2024-03-28T18:00:18.560826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:24.384389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181626+0000","last_change":"2024-03-28T18:00:17.577493+0000","last_active":"2024-03-28T18:00:19.181626+0000","last_peered":"2024-03-28T18:00:19.181626+0000","last_clean":"2024-03-28T18:00:19.181626+0000","last_became_active":"2024-03-28T18:00:17.577228+0000","last_became_peered":"2024-03-28T18:00:17.577228+0000","last_unstale":"2024-03-28T18:00:19.181626+0000","last_undegraded":"2024-03-28T18:00:19.181626+0000","last_fullsized":"2024-03-28T18:00:19.181626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:19:01.893355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962248+0000","last_change":"2024-03-28T18:00:17.603016+0000","last_active":"2024-03-28T18:00:18.962248+0000","last_peered":"2024-03-28T18:00:18.962248+0000","last_clean":"2024-03-28T18:00:18.962248+0000","last_became_active":"2024-03-28T18:00:17.602610+0000","last_became_peered":"2024-03-28T18:00:17.602610+0000","last_unstale":"2024-03-28T18:00:18.962248+0000","last_undegraded":"2024-03-28T18:00:18.962248+0000","last_fullsized":"2024-03-28T18:00:18.962248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:19:01.893355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.961573+0000","last_change":"2024-03-28T18:00:17.596133+0000","last_active":"2024-03-28T18:00:18.961573+0000","last_peered":"2024-03-28T18:00:18.961573+0000","last_clean":"2024-03-28T18:00:18.961573+0000","last_became_active":"2024-03-28T18:00:17.592732+0000","last_became_peered":"2024-03-28T18:00:17.592732+0000","last_unstale":"2024-03-28T18:00:18.961573+0000","last_undegraded":"2024-03-28T18:00:18.961573+0000","last_fullsized":"2024-03-28T18:00:18.961573+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:15:06.195750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182235+0000","last_change":"2024-03-28T18:00:17.593539+0000","last_active":"2024-03-28T18:00:19.182235+0000","last_peered":"2024-03-28T18:00:19.182235+0000","last_clean":"2024-03-28T18:00:19.182235+0000","last_became_active":"2024-03-28T18:00:17.591985+0000","last_became_peered":"2024-03-28T18:00:17.591985+0000","last_unstale":"2024-03-28T18:00:19.182235+0000","last_undegraded":"2024-03-28T18:00:19.182235+0000","last_fullsized":"2024-03-28T18:00:19.182235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:58.530834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181032+0000","last_change":"2024-03-28T18:00:17.576747+0000","last_active":"2024-03-28T18:00:19.181032+0000","last_peered":"2024-03-28T18:00:19.181032+0000","last_clean":"2024-03-28T18:00:19.181032+0000","last_became_active":"2024-03-28T18:00:17.576135+0000","last_became_peered":"2024-03-28T18:00:17.576135+0000","last_unstale":"2024-03-28T18:00:19.181032+0000","last_undegraded":"2024-03-28T18:00:19.181032+0000","last_fullsized":"2024-03-28T18:00:19.181032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:24.555324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962749+0000","last_change":"2024-03-28T18:00:17.595091+0000","last_active":"2024-03-28T18:00:18.962749+0000","last_peered":"2024-03-28T18:00:18.962749+0000","last_clean":"2024-03-28T18:00:18.962749+0000","last_became_active":"2024-03-28T18:00:17.592302+0000","last_became_peered":"2024-03-28T18:00:17.592302+0000","last_unstale":"2024-03-28T18:00:18.962749+0000","last_undegraded":"2024-03-28T18:00:18.962749+0000","last_fullsized":"2024-03-28T18:00:18.962749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:09.743941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562980+0000","last_change":"2024-03-28T18:00:17.575879+0000","last_active":"2024-03-28T18:00:18.562980+0000","last_peered":"2024-03-28T18:00:18.562980+0000","last_clean":"2024-03-28T18:00:18.562980+0000","last_became_active":"2024-03-28T18:00:17.575256+0000","last_became_peered":"2024-03-28T18:00:17.575256+0000","last_unstale":"2024-03-28T18:00:18.562980+0000","last_undegraded":"2024-03-28T18:00:18.562980+0000","last_fullsized":"2024-03-28T18:00:18.562980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:45:42.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.560899+0000","last_change":"2024-03-28T18:00:17.580211+0000","last_active":"2024-03-28T18:00:18.560899+0000","last_peered":"2024-03-28T18:00:18.560899+0000","last_clean":"2024-03-28T18:00:18.560899+0000","last_became_active":"2024-03-28T18:00:17.579630+0000","last_became_peered":"2024-03-28T18:00:17.579630+0000","last_unstale":"2024-03-28T18:00:18.560899+0000","last_undegraded":"2024-03-28T18:00:18.560899+0000","last_fullsized":"2024-03-28T18:00:18.560899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:57.445055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182033+0000","last_change":"2024-03-28T18:00:17.576120+0000","last_active":"2024-03-28T18:00:19.182033+0000","last_peered":"2024-03-28T18:00:19.182033+0000","last_clean":"2024-03-28T18:00:19.182033+0000","last_became_active":"2024-03-28T18:00:17.575817+0000","last_became_peered":"2024-03-28T18:00:17.575817+0000","last_unstale":"2024-03-28T18:00:19.182033+0000","last_undegraded":"2024-03-28T18:00:19.182033+0000","last_fullsized":"2024-03-28T18:00:19.182033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:11:46.184627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562730+0000","last_change":"2024-03-28T18:00:17.582512+0000","last_active":"2024-03-28T18:00:18.562730+0000","last_peered":"2024-03-28T18:00:18.562730+0000","last_clean":"2024-03-28T18:00:18.562730+0000","last_became_active":"2024-03-28T18:00:17.582218+0000","last_became_peered":"2024-03-28T18:00:17.582218+0000","last_unstale":"2024-03-28T18:00:18.562730+0000","last_undegraded":"2024-03-28T18:00:18.562730+0000","last_fullsized":"2024-03-28T18:00:18.562730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:24.555324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962619+0000","last_change":"2024-03-28T18:00:17.600194+0000","last_active":"2024-03-28T18:00:18.962619+0000","last_peered":"2024-03-28T18:00:18.962619+0000","last_clean":"2024-03-28T18:00:18.962619+0000","last_became_active":"2024-03-28T18:00:17.599858+0000","last_became_peered":"2024-03-28T18:00:17.599858+0000","last_unstale":"2024-03-28T18:00:18.962619+0000","last_undegraded":"2024-03-28T18:00:18.962619+0000","last_fullsized":"2024-03-28T18:00:18.962619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:11:46.184627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.963094+0000","last_change":"2024-03-28T18:00:17.600395+0000","last_active":"2024-03-28T18:00:18.963094+0000","last_peered":"2024-03-28T18:00:18.963094+0000","last_clean":"2024-03-28T18:00:18.963094+0000","last_became_active":"2024-03-28T18:00:17.596893+0000","last_became_peered":"2024-03-28T18:00:17.596893+0000","last_unstale":"2024-03-28T18:00:18.963094+0000","last_undegraded":"2024-03-28T18:00:18.963094+0000","last_fullsized":"2024-03-28T18:00:18.963094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:58:44.016114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183102+0000","last_change":"2024-03-28T18:00:17.581494+0000","last_active":"2024-03-28T18:00:19.183102+0000","last_peered":"2024-03-28T18:00:19.183102+0000","last_clean":"2024-03-28T18:00:19.183102+0000","last_became_active":"2024-03-28T18:00:17.581217+0000","last_became_peered":"2024-03-28T18:00:17.581217+0000","last_unstale":"2024-03-28T18:00:19.183102+0000","last_undegraded":"2024-03-28T18:00:19.183102+0000","last_fullsized":"2024-03-28T18:00:19.183102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:08.207407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182319+0000","last_change":"2024-03-28T18:00:17.581759+0000","last_active":"2024-03-28T18:00:19.182319+0000","last_peered":"2024-03-28T18:00:19.182319+0000","last_clean":"2024-03-28T18:00:19.182319+0000","last_became_active":"2024-03-28T18:00:17.581459+0000","last_became_peered":"2024-03-28T18:00:17.581459+0000","last_unstale":"2024-03-28T18:00:19.182319+0000","last_undegraded":"2024-03-28T18:00:19.182319+0000","last_fullsized":"2024-03-28T18:00:19.182319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:09.743941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561649+0000","last_change":"2024-03-28T18:00:17.585946+0000","last_active":"2024-03-28T18:00:18.561649+0000","last_peered":"2024-03-28T18:00:18.561649+0000","last_clean":"2024-03-28T18:00:18.561649+0000","last_became_active":"2024-03-28T18:00:17.584817+0000","last_became_peered":"2024-03-28T18:00:17.584817+0000","last_unstale":"2024-03-28T18:00:18.561649+0000","last_undegraded":"2024-03-28T18:00:18.561649+0000","last_fullsized":"2024-03-28T18:00:18.561649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:22.593553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181142+0000","last_change":"2024-03-28T18:00:17.598965+0000","last_active":"2024-03-28T18:00:19.181142+0000","last_peered":"2024-03-28T18:00:19.181142+0000","last_clean":"2024-03-28T18:00:19.181142+0000","last_became_active":"2024-03-28T18:00:17.598673+0000","last_became_peered":"2024-03-28T18:00:17.598673+0000","last_unstale":"2024-03-28T18:00:19.181142+0000","last_undegraded":"2024-03-28T18:00:19.181142+0000","last_fullsized":"2024-03-28T18:00:19.181142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:28.373593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181888+0000","last_change":"2024-03-28T18:00:17.598232+0000","last_active":"2024-03-28T18:00:19.181888+0000","last_peered":"2024-03-28T18:00:19.181888+0000","last_clean":"2024-03-28T18:00:19.181888+0000","last_became_active":"2024-03-28T18:00:17.597932+0000","last_became_peered":"2024-03-28T18:00:17.597932+0000","last_unstale":"2024-03-28T18:00:19.181888+0000","last_undegraded":"2024-03-28T18:00:19.181888+0000","last_fullsized":"2024-03-28T18:00:19.181888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:47:21.090106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561235+0000","last_change":"2024-03-28T18:00:17.593329+0000","last_active":"2024-03-28T18:00:18.561235+0000","last_peered":"2024-03-28T18:00:18.561235+0000","last_clean":"2024-03-28T18:00:18.561235+0000","last_became_active":"2024-03-28T18:00:17.593025+0000","last_became_peered":"2024-03-28T18:00:17.593025+0000","last_unstale":"2024-03-28T18:00:18.561235+0000","last_undegraded":"2024-03-28T18:00:18.561235+0000","last_fullsized":"2024-03-28T18:00:18.561235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:25.277653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562555+0000","last_change":"2024-03-28T18:00:17.575279+0000","last_active":"2024-03-28T18:00:18.562555+0000","last_peered":"2024-03-28T18:00:18.562555+0000","last_clean":"2024-03-28T18:00:18.562555+0000","last_became_active":"2024-03-28T18:00:17.574758+0000","last_became_peered":"2024-03-28T18:00:17.574758+0000","last_unstale":"2024-03-28T18:00:18.562555+0000","last_undegraded":"2024-03-28T18:00:18.562555+0000","last_fullsized":"2024-03-28T18:00:18.562555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:44.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182631+0000","last_change":"2024-03-28T18:00:17.592334+0000","last_active":"2024-03-28T18:00:19.182631+0000","last_peered":"2024-03-28T18:00:19.182631+0000","last_clean":"2024-03-28T18:00:19.182631+0000","last_became_active":"2024-03-28T18:00:17.592055+0000","last_became_peered":"2024-03-28T18:00:17.592055+0000","last_unstale":"2024-03-28T18:00:19.182631+0000","last_undegraded":"2024-03-28T18:00:19.182631+0000","last_fullsized":"2024-03-28T18:00:19.182631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:32.383519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183441+0000","last_change":"2024-03-28T18:00:17.599418+0000","last_active":"2024-03-28T18:00:19.183441+0000","last_peered":"2024-03-28T18:00:19.183441+0000","last_clean":"2024-03-28T18:00:19.183441+0000","last_became_active":"2024-03-28T18:00:17.599145+0000","last_became_peered":"2024-03-28T18:00:17.599145+0000","last_unstale":"2024-03-28T18:00:19.183441+0000","last_undegraded":"2024-03-28T18:00:19.183441+0000","last_fullsized":"2024-03-28T18:00:19.183441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:14.267177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181904+0000","last_change":"2024-03-28T18:00:17.598613+0000","last_active":"2024-03-28T18:00:19.181904+0000","last_peered":"2024-03-28T18:00:19.181904+0000","last_clean":"2024-03-28T18:00:19.181904+0000","last_became_active":"2024-03-28T18:00:17.597856+0000","last_became_peered":"2024-03-28T18:00:17.597856+0000","last_unstale":"2024-03-28T18:00:19.181904+0000","last_undegraded":"2024-03-28T18:00:19.181904+0000","last_fullsized":"2024-03-28T18:00:19.181904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.364155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.563392+0000","last_change":"2024-03-28T18:00:17.583815+0000","last_active":"2024-03-28T18:00:18.563392+0000","last_peered":"2024-03-28T18:00:18.563392+0000","last_clean":"2024-03-28T18:00:18.563392+0000","last_became_active":"2024-03-28T18:00:17.583526+0000","last_became_peered":"2024-03-28T18:00:17.583526+0000","last_unstale":"2024-03-28T18:00:18.563392+0000","last_undegraded":"2024-03-28T18:00:18.563392+0000","last_fullsized":"2024-03-28T18:00:18.563392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:32.383519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181581+0000","last_change":"2024-03-28T18:00:17.598952+0000","last_active":"2024-03-28T18:00:19.181581+0000","last_peered":"2024-03-28T18:00:19.181581+0000","last_clean":"2024-03-28T18:00:19.181581+0000","last_became_active":"2024-03-28T18:00:17.598614+0000","last_became_peered":"2024-03-28T18:00:17.598614+0000","last_unstale":"2024-03-28T18:00:19.181581+0000","last_undegraded":"2024-03-28T18:00:19.181581+0000","last_fullsized":"2024-03-28T18:00:19.181581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:41.853396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962100+0000","last_change":"2024-03-28T18:00:17.591417+0000","last_active":"2024-03-28T18:00:18.962100+0000","last_peered":"2024-03-28T18:00:18.962100+0000","last_clean":"2024-03-28T18:00:18.962100+0000","last_became_active":"2024-03-28T18:00:17.588144+0000","last_became_peered":"2024-03-28T18:00:17.588144+0000","last_unstale":"2024-03-28T18:00:18.962100+0000","last_undegraded":"2024-03-28T18:00:18.962100+0000","last_fullsized":"2024-03-28T18:00:18.962100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:32.383519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180936+0000","last_change":"2024-03-28T18:00:17.590810+0000","last_active":"2024-03-28T18:00:19.180936+0000","last_peered":"2024-03-28T18:00:19.180936+0000","last_clean":"2024-03-28T18:00:19.180936+0000","last_became_active":"2024-03-28T18:00:17.589590+0000","last_became_peered":"2024-03-28T18:00:17.589590+0000","last_unstale":"2024-03-28T18:00:19.180936+0000","last_undegraded":"2024-03-28T18:00:19.180936+0000","last_fullsized":"2024-03-28T18:00:19.180936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:17:21.722550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.961794+0000","last_change":"2024-03-28T18:00:17.596948+0000","last_active":"2024-03-28T18:00:18.961794+0000","last_peered":"2024-03-28T18:00:18.961794+0000","last_clean":"2024-03-28T18:00:18.961794+0000","last_became_active":"2024-03-28T18:00:17.596601+0000","last_became_peered":"2024-03-28T18:00:17.596601+0000","last_unstale":"2024-03-28T18:00:18.961794+0000","last_undegraded":"2024-03-28T18:00:18.961794+0000","last_fullsized":"2024-03-28T18:00:18.961794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:44.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562624+0000","last_change":"2024-03-28T18:00:17.592508+0000","last_active":"2024-03-28T18:00:18.562624+0000","last_peered":"2024-03-28T18:00:18.562624+0000","last_clean":"2024-03-28T18:00:18.562624+0000","last_became_active":"2024-03-28T18:00:17.592241+0000","last_became_peered":"2024-03-28T18:00:17.592241+0000","last_unstale":"2024-03-28T18:00:18.562624+0000","last_undegraded":"2024-03-28T18:00:18.562624+0000","last_fullsized":"2024-03-28T18:00:18.562624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:41.853396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182418+0000","last_change":"2024-03-28T18:00:16.586041+0000","last_active":"2024-03-28T18:00:19.182418+0000","last_peered":"2024-03-28T18:00:19.182418+0000","last_clean":"2024-03-28T18:00:19.182418+0000","last_became_active":"2024-03-28T18:00:16.585654+0000","last_became_peered":"2024-03-28T18:00:16.585654+0000","last_unstale":"2024-03-28T18:00:19.182418+0000","last_undegraded":"2024-03-28T18:00:19.182418+0000","last_fullsized":"2024-03-28T18:00:19.182418+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182397+0000","last_change":"2024-03-28T18:00:17.598918+0000","last_active":"2024-03-28T18:00:19.182397+0000","last_peered":"2024-03-28T18:00:19.182397+0000","last_clean":"2024-03-28T18:00:19.182397+0000","last_became_active":"2024-03-28T18:00:17.598600+0000","last_became_peered":"2024-03-28T18:00:17.598600+0000","last_unstale":"2024-03-28T18:00:19.182397+0000","last_undegraded":"2024-03-28T18:00:19.182397+0000","last_fullsized":"2024-03-28T18:00:19.182397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.480601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562653+0000","last_change":"2024-03-28T18:00:16.582651+0000","last_active":"2024-03-28T18:00:18.562653+0000","last_peered":"2024-03-28T18:00:18.562653+0000","last_clean":"2024-03-28T18:00:18.562653+0000","last_became_active":"2024-03-28T18:00:16.582248+0000","last_became_peered":"2024-03-28T18:00:16.582248+0000","last_unstale":"2024-03-28T18:00:18.562653+0000","last_undegraded":"2024-03-28T18:00:18.562653+0000","last_fullsized":"2024-03-28T18:00:18.562653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:03:47.228182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182638+0000","last_change":"2024-03-28T18:00:17.595930+0000","last_active":"2024-03-28T18:00:19.182638+0000","last_peered":"2024-03-28T18:00:19.182638+0000","last_clean":"2024-03-28T18:00:19.182638+0000","last_became_active":"2024-03-28T18:00:17.595579+0000","last_became_peered":"2024-03-28T18:00:17.595579+0000","last_unstale":"2024-03-28T18:00:19.182638+0000","last_undegraded":"2024-03-28T18:00:19.182638+0000","last_fullsized":"2024-03-28T18:00:19.182638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:29.120823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182585+0000","last_change":"2024-03-28T18:00:16.588264+0000","last_active":"2024-03-28T18:00:19.182585+0000","last_peered":"2024-03-28T18:00:19.182585+0000","last_clean":"2024-03-28T18:00:19.182585+0000","last_became_active":"2024-03-28T18:00:16.587925+0000","last_became_peered":"2024-03-28T18:00:16.587925+0000","last_unstale":"2024-03-28T18:00:19.182585+0000","last_undegraded":"2024-03-28T18:00:19.182585+0000","last_fullsized":"2024-03-28T18:00:19.182585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:24.261454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182241+0000","last_change":"2024-03-28T18:00:17.598636+0000","last_active":"2024-03-28T18:00:19.182241+0000","last_peered":"2024-03-28T18:00:19.182241+0000","last_clean":"2024-03-28T18:00:19.182241+0000","last_became_active":"2024-03-28T18:00:17.597816+0000","last_became_peered":"2024-03-28T18:00:17.597816+0000","last_unstale":"2024-03-28T18:00:19.182241+0000","last_undegraded":"2024-03-28T18:00:19.182241+0000","last_fullsized":"2024-03-28T18:00:19.182241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:45:42.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561225+0000","last_change":"2024-03-28T18:00:16.577746+0000","last_active":"2024-03-28T18:00:18.561225+0000","last_peered":"2024-03-28T18:00:18.561225+0000","last_clean":"2024-03-28T18:00:18.561225+0000","last_became_active":"2024-03-28T18:00:16.577340+0000","last_became_peered":"2024-03-28T18:00:16.577340+0000","last_unstale":"2024-03-28T18:00:18.561225+0000","last_undegraded":"2024-03-28T18:00:18.561225+0000","last_fullsized":"2024-03-28T18:00:18.561225+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:02.785011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962028+0000","last_change":"2024-03-28T18:00:17.603042+0000","last_active":"2024-03-28T18:00:18.962028+0000","last_peered":"2024-03-28T18:00:18.962028+0000","last_clean":"2024-03-28T18:00:18.962028+0000","last_became_active":"2024-03-28T18:00:17.602658+0000","last_became_peered":"2024-03-28T18:00:17.602658+0000","last_unstale":"2024-03-28T18:00:18.962028+0000","last_undegraded":"2024-03-28T18:00:18.962028+0000","last_fullsized":"2024-03-28T18:00:18.962028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.364155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181817+0000","last_change":"2024-03-28T18:00:16.591523+0000","last_active":"2024-03-28T18:00:19.181817+0000","last_peered":"2024-03-28T18:00:19.181817+0000","last_clean":"2024-03-28T18:00:19.181817+0000","last_became_active":"2024-03-28T18:00:16.591203+0000","last_became_peered":"2024-03-28T18:00:16.591203+0000","last_unstale":"2024-03-28T18:00:19.181817+0000","last_undegraded":"2024-03-28T18:00:19.181817+0000","last_fullsized":"2024-03-28T18:00:19.181817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:38.348966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562909+0000","last_change":"2024-03-28T18:00:17.585403+0000","last_active":"2024-03-28T18:00:18.562909+0000","last_peered":"2024-03-28T18:00:18.562909+0000","last_clean":"2024-03-28T18:00:18.562909+0000","last_became_active":"2024-03-28T18:00:17.584438+0000","last_became_peered":"2024-03-28T18:00:17.584438+0000","last_unstale":"2024-03-28T18:00:18.562909+0000","last_undegraded":"2024-03-28T18:00:18.562909+0000","last_fullsized":"2024-03-28T18:00:18.562909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.364155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181875+0000","last_change":"2024-03-28T18:00:16.591608+0000","last_active":"2024-03-28T18:00:19.181875+0000","last_peered":"2024-03-28T18:00:19.181875+0000","last_clean":"2024-03-28T18:00:19.181875+0000","last_became_active":"2024-03-28T18:00:16.591289+0000","last_became_peered":"2024-03-28T18:00:16.591289+0000","last_unstale":"2024-03-28T18:00:19.181875+0000","last_undegraded":"2024-03-28T18:00:19.181875+0000","last_fullsized":"2024-03-28T18:00:19.181875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:18:21.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180850+0000","last_change":"2024-03-28T18:00:17.588409+0000","last_active":"2024-03-28T18:00:19.180850+0000","last_peered":"2024-03-28T18:00:19.180850+0000","last_clean":"2024-03-28T18:00:19.180850+0000","last_became_active":"2024-03-28T18:00:17.588081+0000","last_became_peered":"2024-03-28T18:00:17.588081+0000","last_unstale":"2024-03-28T18:00:19.180850+0000","last_undegraded":"2024-03-28T18:00:19.180850+0000","last_fullsized":"2024-03-28T18:00:19.180850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:17:21.722550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181412+0000","last_change":"2024-03-28T18:00:16.581766+0000","last_active":"2024-03-28T18:00:19.181412+0000","last_peered":"2024-03-28T18:00:19.181412+0000","last_clean":"2024-03-28T18:00:19.181412+0000","last_became_active":"2024-03-28T18:00:16.581506+0000","last_became_peered":"2024-03-28T18:00:16.581506+0000","last_unstale":"2024-03-28T18:00:19.181412+0000","last_undegraded":"2024-03-28T18:00:19.181412+0000","last_fullsized":"2024-03-28T18:00:19.181412+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:23.143647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.179790+0000","last_change":"2024-03-28T18:00:17.586838+0000","last_active":"2024-03-28T18:00:19.179790+0000","last_peered":"2024-03-28T18:00:19.179790+0000","last_clean":"2024-03-28T18:00:19.179790+0000","last_became_active":"2024-03-28T18:00:17.586548+0000","last_became_peered":"2024-03-28T18:00:17.586548+0000","last_unstale":"2024-03-28T18:00:19.179790+0000","last_undegraded":"2024-03-28T18:00:19.179790+0000","last_fullsized":"2024-03-28T18:00:19.179790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:29.120823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180643+0000","last_change":"2024-03-28T18:00:16.589309+0000","last_active":"2024-03-28T18:00:19.180643+0000","last_peered":"2024-03-28T18:00:19.180643+0000","last_clean":"2024-03-28T18:00:19.180643+0000","last_became_active":"2024-03-28T18:00:16.588995+0000","last_became_peered":"2024-03-28T18:00:16.588995+0000","last_unstale":"2024-03-28T18:00:19.180643+0000","last_undegraded":"2024-03-28T18:00:19.180643+0000","last_fullsized":"2024-03-28T18:00:19.180643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:36.237216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.563780+0000","last_change":"2024-03-28T18:00:17.587301+0000","last_active":"2024-03-28T18:00:18.563780+0000","last_peered":"2024-03-28T18:00:18.563780+0000","last_clean":"2024-03-28T18:00:18.563780+0000","last_became_active":"2024-03-28T18:00:17.586987+0000","last_became_peered":"2024-03-28T18:00:17.586987+0000","last_unstale":"2024-03-28T18:00:18.563780+0000","last_undegraded":"2024-03-28T18:00:18.563780+0000","last_fullsized":"2024-03-28T18:00:18.563780+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:11:46.184627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183086+0000","last_change":"2024-03-28T18:00:16.588159+0000","last_active":"2024-03-28T18:00:19.183086+0000","last_peered":"2024-03-28T18:00:19.183086+0000","last_clean":"2024-03-28T18:00:19.183086+0000","last_became_active":"2024-03-28T18:00:16.587891+0000","last_became_peered":"2024-03-28T18:00:16.587891+0000","last_unstale":"2024-03-28T18:00:19.183086+0000","last_undegraded":"2024-03-28T18:00:19.183086+0000","last_fullsized":"2024-03-28T18:00:19.183086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:27.357394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183147+0000","last_change":"2024-03-28T18:00:17.585494+0000","last_active":"2024-03-28T18:00:19.183147+0000","last_peered":"2024-03-28T18:00:19.183147+0000","last_clean":"2024-03-28T18:00:19.183147+0000","last_became_active":"2024-03-28T18:00:17.585219+0000","last_became_peered":"2024-03-28T18:00:17.585219+0000","last_unstale":"2024-03-28T18:00:19.183147+0000","last_undegraded":"2024-03-28T18:00:19.183147+0000","last_fullsized":"2024-03-28T18:00:19.183147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:52.438314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562763+0000","last_change":"2024-03-28T18:00:16.590209+0000","last_active":"2024-03-28T18:00:18.562763+0000","last_peered":"2024-03-28T18:00:18.562763+0000","last_clean":"2024-03-28T18:00:18.562763+0000","last_became_active":"2024-03-28T18:00:16.589874+0000","last_became_peered":"2024-03-28T18:00:16.589874+0000","last_unstale":"2024-03-28T18:00:18.562763+0000","last_undegraded":"2024-03-28T18:00:18.562763+0000","last_fullsized":"2024-03-28T18:00:18.562763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182398+0000","last_change":"2024-03-28T18:00:17.599413+0000","last_active":"2024-03-28T18:00:19.182398+0000","last_peered":"2024-03-28T18:00:19.182398+0000","last_clean":"2024-03-28T18:00:19.182398+0000","last_became_active":"2024-03-28T18:00:17.599134+0000","last_became_peered":"2024-03-28T18:00:17.599134+0000","last_unstale":"2024-03-28T18:00:19.182398+0000","last_undegraded":"2024-03-28T18:00:19.182398+0000","last_fullsized":"2024-03-28T18:00:19.182398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:58:44.016114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.563753+0000","last_change":"2024-03-28T18:00:16.583925+0000","last_active":"2024-03-28T18:00:18.563753+0000","last_peered":"2024-03-28T18:00:18.563753+0000","last_clean":"2024-03-28T18:00:18.563753+0000","last_became_active":"2024-03-28T18:00:16.583635+0000","last_became_peered":"2024-03-28T18:00:16.583635+0000","last_unstale":"2024-03-28T18:00:18.563753+0000","last_undegraded":"2024-03-28T18:00:18.563753+0000","last_fullsized":"2024-03-28T18:00:18.563753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:04.966170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.563464+0000","last_change":"2024-03-28T18:00:17.580424+0000","last_active":"2024-03-28T18:00:18.563464+0000","last_peered":"2024-03-28T18:00:18.563464+0000","last_clean":"2024-03-28T18:00:18.563464+0000","last_became_active":"2024-03-28T18:00:17.579776+0000","last_became_peered":"2024-03-28T18:00:17.579776+0000","last_unstale":"2024-03-28T18:00:18.563464+0000","last_undegraded":"2024-03-28T18:00:18.563464+0000","last_fullsized":"2024-03-28T18:00:18.563464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:19:01.893355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182286+0000","last_change":"2024-03-28T18:00:16.593104+0000","last_active":"2024-03-28T18:00:19.182286+0000","last_peered":"2024-03-28T18:00:19.182286+0000","last_clean":"2024-03-28T18:00:19.182286+0000","last_became_active":"2024-03-28T18:00:16.592771+0000","last_became_peered":"2024-03-28T18:00:16.592771+0000","last_unstale":"2024-03-28T18:00:19.182286+0000","last_undegraded":"2024-03-28T18:00:19.182286+0000","last_fullsized":"2024-03-28T18:00:19.182286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:17:20.706351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.179714+0000","last_change":"2024-03-28T18:00:17.585557+0000","last_active":"2024-03-28T18:00:19.179714+0000","last_peered":"2024-03-28T18:00:19.179714+0000","last_clean":"2024-03-28T18:00:19.179714+0000","last_became_active":"2024-03-28T18:00:17.585227+0000","last_became_peered":"2024-03-28T18:00:17.585227+0000","last_unstale":"2024-03-28T18:00:19.179714+0000","last_undegraded":"2024-03-28T18:00:19.179714+0000","last_fullsized":"2024-03-28T18:00:19.179714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:58.530834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182083+0000","last_change":"2024-03-28T18:00:16.584849+0000","last_active":"2024-03-28T18:00:19.182083+0000","last_peered":"2024-03-28T18:00:19.182083+0000","last_clean":"2024-03-28T18:00:19.182083+0000","last_became_active":"2024-03-28T18:00:16.584543+0000","last_became_peered":"2024-03-28T18:00:16.584543+0000","last_unstale":"2024-03-28T18:00:19.182083+0000","last_undegraded":"2024-03-28T18:00:19.182083+0000","last_fullsized":"2024-03-28T18:00:19.182083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:23.939804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181897+0000","last_change":"2024-03-28T18:00:17.568209+0000","last_active":"2024-03-28T18:00:19.181897+0000","last_peered":"2024-03-28T18:00:19.181897+0000","last_clean":"2024-03-28T18:00:19.181897+0000","last_became_active":"2024-03-28T18:00:17.566176+0000","last_became_peered":"2024-03-28T18:00:17.566176+0000","last_unstale":"2024-03-28T18:00:19.181897+0000","last_undegraded":"2024-03-28T18:00:19.181897+0000","last_fullsized":"2024-03-28T18:00:19.181897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:12:03.104842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.179939+0000","last_change":"2024-03-28T18:00:16.580448+0000","last_active":"2024-03-28T18:00:19.179939+0000","last_peered":"2024-03-28T18:00:19.179939+0000","last_clean":"2024-03-28T18:00:19.179939+0000","last_became_active":"2024-03-28T18:00:16.579745+0000","last_became_peered":"2024-03-28T18:00:16.579745+0000","last_unstale":"2024-03-28T18:00:19.179939+0000","last_undegraded":"2024-03-28T18:00:19.179939+0000","last_fullsized":"2024-03-28T18:00:19.179939+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:15.738771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962088+0000","last_change":"2024-03-28T18:00:17.603331+0000","last_active":"2024-03-28T18:00:18.962088+0000","last_peered":"2024-03-28T18:00:18.962088+0000","last_clean":"2024-03-28T18:00:18.962088+0000","last_became_active":"2024-03-28T18:00:17.603044+0000","last_became_peered":"2024-03-28T18:00:17.603044+0000","last_unstale":"2024-03-28T18:00:18.962088+0000","last_undegraded":"2024-03-28T18:00:18.962088+0000","last_fullsized":"2024-03-28T18:00:18.962088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:45:42.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181336+0000","last_change":"2024-03-28T18:00:16.587497+0000","last_active":"2024-03-28T18:00:19.181336+0000","last_peered":"2024-03-28T18:00:19.181336+0000","last_clean":"2024-03-28T18:00:19.181336+0000","last_became_active":"2024-03-28T18:00:16.586180+0000","last_became_peered":"2024-03-28T18:00:16.586180+0000","last_unstale":"2024-03-28T18:00:19.181336+0000","last_undegraded":"2024-03-28T18:00:19.181336+0000","last_fullsized":"2024-03-28T18:00:19.181336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:03:47.228182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181654+0000","last_change":"2024-03-28T18:00:17.598780+0000","last_active":"2024-03-28T18:00:19.181654+0000","last_peered":"2024-03-28T18:00:19.181654+0000","last_clean":"2024-03-28T18:00:19.181654+0000","last_became_active":"2024-03-28T18:00:17.598255+0000","last_became_peered":"2024-03-28T18:00:17.598255+0000","last_unstale":"2024-03-28T18:00:19.181654+0000","last_undegraded":"2024-03-28T18:00:19.181654+0000","last_fullsized":"2024-03-28T18:00:19.181654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:25.277653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181030+0000","last_change":"2024-03-28T18:00:16.581297+0000","last_active":"2024-03-28T18:00:19.181030+0000","last_peered":"2024-03-28T18:00:19.181030+0000","last_clean":"2024-03-28T18:00:19.181030+0000","last_became_active":"2024-03-28T18:00:16.580937+0000","last_became_peered":"2024-03-28T18:00:16.580937+0000","last_unstale":"2024-03-28T18:00:19.181030+0000","last_undegraded":"2024-03-28T18:00:19.181030+0000","last_fullsized":"2024-03-28T18:00:19.181030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:04.966170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181347+0000","last_change":"2024-03-28T18:00:17.595308+0000","last_active":"2024-03-28T18:00:19.181347+0000","last_peered":"2024-03-28T18:00:19.181347+0000","last_clean":"2024-03-28T18:00:19.181347+0000","last_became_active":"2024-03-28T18:00:17.594142+0000","last_became_peered":"2024-03-28T18:00:17.594142+0000","last_unstale":"2024-03-28T18:00:19.181347+0000","last_undegraded":"2024-03-28T18:00:19.181347+0000","last_fullsized":"2024-03-28T18:00:19.181347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.480601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182816+0000","last_change":"2024-03-28T18:00:16.589262+0000","last_active":"2024-03-28T18:00:19.182816+0000","last_peered":"2024-03-28T18:00:19.182816+0000","last_clean":"2024-03-28T18:00:19.182816+0000","last_became_active":"2024-03-28T18:00:16.588786+0000","last_became_peered":"2024-03-28T18:00:16.588786+0000","last_unstale":"2024-03-28T18:00:19.182816+0000","last_undegraded":"2024-03-28T18:00:19.182816+0000","last_fullsized":"2024-03-28T18:00:19.182816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:03:47.228182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.564017+0000","last_change":"2024-03-28T18:00:17.595261+0000","last_active":"2024-03-28T18:00:18.564017+0000","last_peered":"2024-03-28T18:00:18.564017+0000","last_clean":"2024-03-28T18:00:18.564017+0000","last_became_active":"2024-03-28T18:00:17.594973+0000","last_became_peered":"2024-03-28T18:00:17.594973+0000","last_unstale":"2024-03-28T18:00:18.564017+0000","last_undegraded":"2024-03-28T18:00:18.564017+0000","last_fullsized":"2024-03-28T18:00:18.564017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:09.743941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181639+0000","last_change":"2024-03-28T18:00:16.587380+0000","last_active":"2024-03-28T18:00:19.181639+0000","last_peered":"2024-03-28T18:00:19.181639+0000","last_clean":"2024-03-28T18:00:19.181639+0000","last_became_active":"2024-03-28T18:00:16.586842+0000","last_became_peered":"2024-03-28T18:00:16.586842+0000","last_unstale":"2024-03-28T18:00:19.181639+0000","last_undegraded":"2024-03-28T18:00:19.181639+0000","last_fullsized":"2024-03-28T18:00:19.181639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:37.540343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180440+0000","last_change":"2024-03-28T18:00:17.598488+0000","last_active":"2024-03-28T18:00:19.180440+0000","last_peered":"2024-03-28T18:00:19.180440+0000","last_clean":"2024-03-28T18:00:19.180440+0000","last_became_active":"2024-03-28T18:00:17.597833+0000","last_became_peered":"2024-03-28T18:00:17.597833+0000","last_unstale":"2024-03-28T18:00:19.180440+0000","last_undegraded":"2024-03-28T18:00:19.180440+0000","last_fullsized":"2024-03-28T18:00:19.180440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:44.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.183233+0000","last_change":"2024-03-28T18:00:16.581961+0000","last_active":"2024-03-28T18:00:19.183233+0000","last_peered":"2024-03-28T18:00:19.183233+0000","last_clean":"2024-03-28T18:00:19.183233+0000","last_became_active":"2024-03-28T18:00:16.581694+0000","last_became_peered":"2024-03-28T18:00:16.581694+0000","last_unstale":"2024-03-28T18:00:19.183233+0000","last_undegraded":"2024-03-28T18:00:19.183233+0000","last_fullsized":"2024-03-28T18:00:19.183233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:56.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561743+0000","last_change":"2024-03-28T18:00:17.588116+0000","last_active":"2024-03-28T18:00:18.561743+0000","last_peered":"2024-03-28T18:00:18.561743+0000","last_clean":"2024-03-28T18:00:18.561743+0000","last_became_active":"2024-03-28T18:00:17.587575+0000","last_became_peered":"2024-03-28T18:00:17.587575+0000","last_unstale":"2024-03-28T18:00:18.561743+0000","last_undegraded":"2024-03-28T18:00:18.561743+0000","last_fullsized":"2024-03-28T18:00:18.561743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:03.564784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182611+0000","last_change":"2024-03-28T18:00:16.590452+0000","last_active":"2024-03-28T18:00:19.182611+0000","last_peered":"2024-03-28T18:00:19.182611+0000","last_clean":"2024-03-28T18:00:19.182611+0000","last_became_active":"2024-03-28T18:00:16.590148+0000","last_became_peered":"2024-03-28T18:00:16.590148+0000","last_unstale":"2024-03-28T18:00:19.182611+0000","last_undegraded":"2024-03-28T18:00:19.182611+0000","last_fullsized":"2024-03-28T18:00:19.182611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:23.368190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.560623+0000","last_change":"2024-03-28T18:00:17.600197+0000","last_active":"2024-03-28T18:00:18.560623+0000","last_peered":"2024-03-28T18:00:18.560623+0000","last_clean":"2024-03-28T18:00:18.560623+0000","last_became_active":"2024-03-28T18:00:17.599846+0000","last_became_peered":"2024-03-28T18:00:17.599846+0000","last_unstale":"2024-03-28T18:00:18.560623+0000","last_undegraded":"2024-03-28T18:00:18.560623+0000","last_fullsized":"2024-03-28T18:00:18.560623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:30.334911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562010+0000","last_change":"2024-03-28T18:00:16.574301+0000","last_active":"2024-03-28T18:00:18.562010+0000","last_peered":"2024-03-28T18:00:18.562010+0000","last_clean":"2024-03-28T18:00:18.562010+0000","last_became_active":"2024-03-28T18:00:16.573671+0000","last_became_peered":"2024-03-28T18:00:16.573671+0000","last_unstale":"2024-03-28T18:00:18.562010+0000","last_undegraded":"2024-03-28T18:00:18.562010+0000","last_fullsized":"2024-03-28T18:00:18.562010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561291+0000","last_change":"2024-03-28T18:00:17.578376+0000","last_active":"2024-03-28T18:00:18.561291+0000","last_peered":"2024-03-28T18:00:18.561291+0000","last_clean":"2024-03-28T18:00:18.561291+0000","last_became_active":"2024-03-28T18:00:17.577914+0000","last_became_peered":"2024-03-28T18:00:17.577914+0000","last_unstale":"2024-03-28T18:00:18.561291+0000","last_undegraded":"2024-03-28T18:00:18.561291+0000","last_fullsized":"2024-03-28T18:00:18.561291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:52:39.365165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182228+0000","last_change":"2024-03-28T18:00:16.584421+0000","last_active":"2024-03-28T18:00:19.182228+0000","last_peered":"2024-03-28T18:00:19.182228+0000","last_clean":"2024-03-28T18:00:19.182228+0000","last_became_active":"2024-03-28T18:00:16.584113+0000","last_became_peered":"2024-03-28T18:00:16.584113+0000","last_unstale":"2024-03-28T18:00:19.182228+0000","last_undegraded":"2024-03-28T18:00:19.182228+0000","last_fullsized":"2024-03-28T18:00:19.182228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561239+0000","last_change":"2024-03-28T18:00:17.576515+0000","last_active":"2024-03-28T18:00:18.561239+0000","last_peered":"2024-03-28T18:00:18.561239+0000","last_clean":"2024-03-28T18:00:18.561239+0000","last_became_active":"2024-03-28T18:00:17.576185+0000","last_became_peered":"2024-03-28T18:00:17.576185+0000","last_unstale":"2024-03-28T18:00:18.561239+0000","last_undegraded":"2024-03-28T18:00:18.561239+0000","last_fullsized":"2024-03-28T18:00:18.561239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:38.556542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180932+0000","last_change":"2024-03-28T18:00:16.578260+0000","last_active":"2024-03-28T18:00:19.180932+0000","last_peered":"2024-03-28T18:00:19.180932+0000","last_clean":"2024-03-28T18:00:19.180932+0000","last_became_active":"2024-03-28T18:00:16.577824+0000","last_became_peered":"2024-03-28T18:00:16.577824+0000","last_unstale":"2024-03-28T18:00:19.180932+0000","last_undegraded":"2024-03-28T18:00:19.180932+0000","last_fullsized":"2024-03-28T18:00:19.180932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:27.046784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.961935+0000","last_change":"2024-03-28T18:00:17.602995+0000","last_active":"2024-03-28T18:00:18.961935+0000","last_peered":"2024-03-28T18:00:18.961935+0000","last_clean":"2024-03-28T18:00:18.961935+0000","last_became_active":"2024-03-28T18:00:17.602595+0000","last_became_peered":"2024-03-28T18:00:17.602595+0000","last_unstale":"2024-03-28T18:00:18.961935+0000","last_undegraded":"2024-03-28T18:00:18.961935+0000","last_fullsized":"2024-03-28T18:00:18.961935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:10:24.555324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562288+0000","last_change":"2024-03-28T18:00:16.584736+0000","last_active":"2024-03-28T18:00:18.562288+0000","last_peered":"2024-03-28T18:00:18.562288+0000","last_clean":"2024-03-28T18:00:18.562288+0000","last_became_active":"2024-03-28T18:00:16.584455+0000","last_became_peered":"2024-03-28T18:00:16.584455+0000","last_unstale":"2024-03-28T18:00:18.562288+0000","last_undegraded":"2024-03-28T18:00:18.562288+0000","last_fullsized":"2024-03-28T18:00:18.562288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:26:29.402283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.561504+0000","last_change":"2024-03-28T18:00:17.583958+0000","last_active":"2024-03-28T18:00:18.561504+0000","last_peered":"2024-03-28T18:00:18.561504+0000","last_clean":"2024-03-28T18:00:18.561504+0000","last_became_active":"2024-03-28T18:00:17.583685+0000","last_became_peered":"2024-03-28T18:00:17.583685+0000","last_unstale":"2024-03-28T18:00:18.561504+0000","last_undegraded":"2024-03-28T18:00:18.561504+0000","last_fullsized":"2024-03-28T18:00:18.561504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:24.956003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182926+0000","last_change":"2024-03-28T18:00:16.583053+0000","last_active":"2024-03-28T18:00:19.182926+0000","last_peered":"2024-03-28T18:00:19.182926+0000","last_clean":"2024-03-28T18:00:19.182926+0000","last_became_active":"2024-03-28T18:00:16.582775+0000","last_became_peered":"2024-03-28T18:00:16.582775+0000","last_unstale":"2024-03-28T18:00:19.182926+0000","last_undegraded":"2024-03-28T18:00:19.182926+0000","last_fullsized":"2024-03-28T18:00:19.182926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:26:29.402283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.564058+0000","last_change":"2024-03-28T18:00:17.583244+0000","last_active":"2024-03-28T18:00:18.564058+0000","last_peered":"2024-03-28T18:00:18.564058+0000","last_clean":"2024-03-28T18:00:18.564058+0000","last_became_active":"2024-03-28T18:00:17.582912+0000","last_became_peered":"2024-03-28T18:00:17.582912+0000","last_unstale":"2024-03-28T18:00:18.564058+0000","last_undegraded":"2024-03-28T18:00:18.564058+0000","last_fullsized":"2024-03-28T18:00:18.564058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:58:44.016114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562879+0000","last_change":"2024-03-28T18:00:16.575792+0000","last_active":"2024-03-28T18:00:18.562879+0000","last_peered":"2024-03-28T18:00:18.562879+0000","last_clean":"2024-03-28T18:00:18.562879+0000","last_became_active":"2024-03-28T18:00:16.575475+0000","last_became_peered":"2024-03-28T18:00:16.575475+0000","last_unstale":"2024-03-28T18:00:18.562879+0000","last_undegraded":"2024-03-28T18:00:18.562879+0000","last_fullsized":"2024-03-28T18:00:18.562879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562184+0000","last_change":"2024-03-28T18:00:17.580259+0000","last_active":"2024-03-28T18:00:18.562184+0000","last_peered":"2024-03-28T18:00:18.562184+0000","last_clean":"2024-03-28T18:00:18.562184+0000","last_became_active":"2024-03-28T18:00:17.579922+0000","last_became_peered":"2024-03-28T18:00:17.579922+0000","last_unstale":"2024-03-28T18:00:18.562184+0000","last_undegraded":"2024-03-28T18:00:18.562184+0000","last_fullsized":"2024-03-28T18:00:18.562184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:28.373593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180705+0000","last_change":"2024-03-28T18:00:16.576446+0000","last_active":"2024-03-28T18:00:19.180705+0000","last_peered":"2024-03-28T18:00:19.180705+0000","last_clean":"2024-03-28T18:00:19.180705+0000","last_became_active":"2024-03-28T18:00:16.575444+0000","last_became_peered":"2024-03-28T18:00:16.575444+0000","last_unstale":"2024-03-28T18:00:19.180705+0000","last_undegraded":"2024-03-28T18:00:19.180705+0000","last_fullsized":"2024-03-28T18:00:19.180705+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.182663+0000","last_change":"2024-03-28T18:00:17.579645+0000","last_active":"2024-03-28T18:00:19.182663+0000","last_peered":"2024-03-28T18:00:19.182663+0000","last_clean":"2024-03-28T18:00:19.182663+0000","last_became_active":"2024-03-28T18:00:17.579242+0000","last_became_peered":"2024-03-28T18:00:17.579242+0000","last_unstale":"2024-03-28T18:00:19.182663+0000","last_undegraded":"2024-03-28T18:00:19.182663+0000","last_fullsized":"2024-03-28T18:00:19.182663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:41.853396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562268+0000","last_change":"2024-03-28T18:00:16.573496+0000","last_active":"2024-03-28T18:00:18.562268+0000","last_peered":"2024-03-28T18:00:18.562268+0000","last_clean":"2024-03-28T18:00:18.562268+0000","last_became_active":"2024-03-28T18:00:16.573041+0000","last_became_peered":"2024-03-28T18:00:16.573041+0000","last_unstale":"2024-03-28T18:00:18.562268+0000","last_undegraded":"2024-03-28T18:00:18.562268+0000","last_fullsized":"2024-03-28T18:00:18.562268+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:04.495870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.562184+0000","last_change":"2024-03-28T18:00:17.583682+0000","last_active":"2024-03-28T18:00:18.562184+0000","last_peered":"2024-03-28T18:00:18.562184+0000","last_clean":"2024-03-28T18:00:18.562184+0000","last_became_active":"2024-03-28T18:00:17.583304+0000","last_became_peered":"2024-03-28T18:00:17.583304+0000","last_unstale":"2024-03-28T18:00:18.562184+0000","last_undegraded":"2024-03-28T18:00:18.562184+0000","last_fullsized":"2024-03-28T18:00:18.562184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.480601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180172+0000","last_change":"2024-03-28T18:00:16.581384+0000","last_active":"2024-03-28T18:00:19.180172+0000","last_peered":"2024-03-28T18:00:19.180172+0000","last_clean":"2024-03-28T18:00:19.180172+0000","last_became_active":"2024-03-28T18:00:16.581030+0000","last_became_peered":"2024-03-28T18:00:16.581030+0000","last_unstale":"2024-03-28T18:00:19.180172+0000","last_undegraded":"2024-03-28T18:00:19.180172+0000","last_fullsized":"2024-03-28T18:00:19.180172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:56.546713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180615+0000","last_change":"2024-03-28T18:00:17.578867+0000","last_active":"2024-03-28T18:00:19.180615+0000","last_peered":"2024-03-28T18:00:19.180615+0000","last_clean":"2024-03-28T18:00:19.180615+0000","last_became_active":"2024-03-28T18:00:17.578589+0000","last_became_peered":"2024-03-28T18:00:17.578589+0000","last_unstale":"2024-03-28T18:00:19.180615+0000","last_undegraded":"2024-03-28T18:00:19.180615+0000","last_fullsized":"2024-03-28T18:00:19.180615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:32.383519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181163+0000","last_change":"2024-03-28T18:00:16.582503+0000","last_active":"2024-03-28T18:00:19.181163+0000","last_peered":"2024-03-28T18:00:19.181163+0000","last_clean":"2024-03-28T18:00:19.181163+0000","last_became_active":"2024-03-28T18:00:16.581354+0000","last_became_peered":"2024-03-28T18:00:16.581354+0000","last_unstale":"2024-03-28T18:00:19.181163+0000","last_undegraded":"2024-03-28T18:00:19.181163+0000","last_fullsized":"2024-03-28T18:00:19.181163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:29.318712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180882+0000","last_change":"2024-03-28T18:00:17.590127+0000","last_active":"2024-03-28T18:00:19.180882+0000","last_peered":"2024-03-28T18:00:19.180882+0000","last_clean":"2024-03-28T18:00:19.180882+0000","last_became_active":"2024-03-28T18:00:17.589834+0000","last_became_peered":"2024-03-28T18:00:17.589834+0000","last_unstale":"2024-03-28T18:00:19.180882+0000","last_undegraded":"2024-03-28T18:00:19.180882+0000","last_fullsized":"2024-03-28T18:00:19.180882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:25.277653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.180604+0000","last_change":"2024-03-28T18:00:16.587890+0000","last_active":"2024-03-28T18:00:19.180604+0000","last_peered":"2024-03-28T18:00:19.180604+0000","last_clean":"2024-03-28T18:00:19.180604+0000","last_became_active":"2024-03-28T18:00:16.587564+0000","last_became_peered":"2024-03-28T18:00:16.587564+0000","last_unstale":"2024-03-28T18:00:19.180604+0000","last_undegraded":"2024-03-28T18:00:19.180604+0000","last_fullsized":"2024-03-28T18:00:19.180604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:26:29.402283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962467+0000","last_change":"2024-03-28T18:00:17.593590+0000","last_active":"2024-03-28T18:00:18.962467+0000","last_peered":"2024-03-28T18:00:18.962467+0000","last_clean":"2024-03-28T18:00:18.962467+0000","last_became_active":"2024-03-28T18:00:17.593232+0000","last_became_peered":"2024-03-28T18:00:17.593232+0000","last_unstale":"2024-03-28T18:00:18.962467+0000","last_undegraded":"2024-03-28T18:00:18.962467+0000","last_fullsized":"2024-03-28T18:00:18.962467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:16.533783+0000","last_clean_scrub_stamp":"2024-03-28T18:00:16.533783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:12:03.104842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:19.181511+0000","last_change":"2024-03-28T18:00:16.589804+0000","last_active":"2024-03-28T18:00:19.181511+0000","last_peered":"2024-03-28T18:00:19.181511+0000","last_clean":"2024-03-28T18:00:19.181511+0000","last_became_active":"2024-03-28T18:00:16.589477+0000","last_became_peered":"2024-03-28T18:00:16.589477+0000","last_unstale":"2024-03-28T18:00:19.181511+0000","last_undegraded":"2024-03-28T18:00:19.181511+0000","last_fullsized":"2024-03-28T18:00:19.181511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:00:15.517584+0000","last_clean_scrub_stamp":"2024-03-28T18:00:15.517584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:02.548585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T18:00:18.962436+0000","last_change":"2024-03-28T17:58:43.778584+0000","last_active":"2024-03-28T18:00:18.962436+0000","last_peered":"2024-03-28T18:00:18.962436+0000","last_clean":"2024-03-28T18:00:18.962436+0000","last_became_active":"2024-03-28T17:58:43.777849+0000","last_became_peered":"2024-03-28T17:58:43.777849+0000","last_unstale":"2024-03-28T18:00:18.962436+0000","last_undegraded":"2024-03-28T18:00:18.962436+0000","last_fullsized":"2024-03-28T18:00:18.962436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:42.596664+0000","last_clean_scrub_stamp":"2024-03-28T17:58:42.596664+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:57.018033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:00:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T18:00:27.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T18:00:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.157+0000 7f480fe00640 1 Processor -- start 2024-03-28T18:00:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.158+0000 7f480fe00640 1 -- start start 2024-03-28T18:00:28.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480fe00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4808144700 0x7f480806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806e740 0x7f48080b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480fe00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48080b3b50 con 0x7f480806dba0 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080b4760 con 0x7f480806e740 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480fe00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080b3e30 con 0x7f4808144700 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.161+0000 7f480de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806e740 0x7f48080b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806e740 0x7f48080b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60348/0 (socket says 172.21.15.62:60348) 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 -- 172.21.15.62:0/4128917135 learned_addr learned my addr 172.21.15.62:0/4128917135 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480d400640 1 --2- 172.21.15.62:0/4128917135 >> v2:172.21.15.81:3300/0 conn(0x7f4808144700 0x7f480806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 -- 172.21.15.62:0/4128917135 >> v2:172.21.15.81:3300/0 conn(0x7f4808144700 msgr2=0x7f480806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 --2- 172.21.15.62:0/4128917135 >> v2:172.21.15.81:3300/0 conn(0x7f4808144700 0x7f480806f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 -- 172.21.15.62:0/4128917135 >> v1:172.21.15.81:6789/0 conn(0x7f480806dba0 legacy=0x7f480806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:28.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.162+0000 7f480de00640 1 -- 172.21.15.62:0/4128917135 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080b4110 con 0x7f480806e740 2024-03-28T18:00:28.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.163+0000 7f480de00640 1 --2- 172.21.15.62:0/4128917135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806e740 0x7f48080b4bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f48080717a0 tx=0x7f480002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fb62f1724f8bee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f4807e00640 1 -- 172.21.15.62:0/4128917135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480002c9f0 con 0x7f480806e740 2024-03-28T18:00:28.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f4807e00640 1 -- 172.21.15.62:0/4128917135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480002cb90 con 0x7f480806e740 2024-03-28T18:00:28.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f4807e00640 1 -- 172.21.15.62:0/4128917135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000343f0 con 0x7f480806e740 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 -- 172.21.15.62:0/4128917135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806e740 msgr2=0x7f48080b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 --2- 172.21.15.62:0/4128917135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806e740 0x7f48080b4bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f48080717a0 tx=0x7f480002be50 comp rx=0 tx=0).stop 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 -- 172.21.15.62:0/4128917135 shutdown_connections 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 --2- 172.21.15.62:0/4128917135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806e740 0x7f48080b4bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 --2- 172.21.15.62:0/4128917135 >> v2:172.21.15.81:3300/0 conn(0x7f4808144700 0x7f480806f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.164+0000 7f480fe00640 1 -- 172.21.15.62:0/4128917135 >> 172.21.15.62:0/4128917135 conn(0x7f480806b0a0 msgr2=0x7f480806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.165+0000 7f480fe00640 1 -- 172.21.15.62:0/4128917135 shutdown_connections 2024-03-28T18:00:28.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.165+0000 7f480fe00640 1 -- 172.21.15.62:0/4128917135 wait complete. 2024-03-28T18:00:28.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.166+0000 7f480fe00640 1 Processor -- start 2024-03-28T18:00:28.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.167+0000 7f480fe00640 1 -- start start 2024-03-28T18:00:28.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.167+0000 7f480fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480fe00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480806e740 0x7f48081e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4808144700 0x7f48081301a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080cde70 con 0x7f480806dba0 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480fe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480806fb00 con 0x7f4808144700 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480fe00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080cdff0 con 0x7f480806e740 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60350/0 (socket says 172.21.15.62:60350) 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 -- 172.21.15.62:0/184355500 learned_addr learned my addr 172.21.15.62:0/184355500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 -- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480806e740 msgr2=0x7f48081e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480806e740 0x7f48081e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480de00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4808144700 0x7f48081301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 -- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4808144700 msgr2=0x7f48081301a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4808144700 0x7f48081301a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.168+0000 7f480d400640 1 -- 172.21.15.62:0/184355500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48081e9080 con 0x7f480806dba0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f480d400640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00d830 tx=0x7f47fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00b6c0 con 0x7f480806dba0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc019070 con 0x7f480806dba0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00ba80 con 0x7f480806dba0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48081e9360 con 0x7f480806dba0 2024-03-28T18:00:28.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.169+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48081e9920 con 0x7f480806dba0 2024-03-28T18:00:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.170+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080687d0 con 0x7f480806dba0 2024-03-28T18:00:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.171+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f47fc01e460 con 0x7f480806dba0 2024-03-28T18:00:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.172+0000 7f4805600640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 0x7f47e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.172+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47fc08b1e0 con 0x7f480806dba0 2024-03-28T18:00:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.173+0000 7f480ca00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 0x7f47e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.174+0000 7f480ca00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 0x7f47e806e090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.177+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f47fc059c40 con 0x7f480806dba0 2024-03-28T18:00:28.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.455+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f480806be70 con 0x7f480806dba0 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.460+0000 7f4805600640 1 -- 172.21.15.62:0/184355500 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f47fc059540 con 0x7f480806dba0 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCcrwVm9FlXOxAAfVJXlLgfSVrvwPjHrZ1V6Q== 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T18:00:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-03-28T18:00:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 msgr2=0x7f47e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 0x7f47e806e090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4006bc0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 msgr2=0x7f480815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00d830 tx=0x7f47fc00dcf0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 shutdown_connections 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f47e806bbd0 0x7f47e806e090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4808144700 0x7f48081301a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f480806e740 0x7f48081e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 --2- 172.21.15.62:0/184355500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806dba0 0x7f480815a5b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.463+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 >> 172.21.15.62:0/184355500 conn(0x7f480806b0a0 msgr2=0x7f480806e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.464+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 shutdown_connections 2024-03-28T18:00:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.464+0000 7f480fe00640 1 -- 172.21.15.62:0/184355500 wait complete. 2024-03-28T18:00:28.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:00:28.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.707+0000 7f0936400640 1 Processor -- start 2024-03-28T18:00:28.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.707+0000 7f0936400640 1 -- start start 2024-03-28T18:00:28.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0936400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09301456d0 0x7f093015f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0936400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f093015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0936400640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09300cebb0 con 0x7f093006cd80 2024-03-28T18:00:28.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0936400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09300cee90 con 0x7f09301456d0 2024-03-28T18:00:28.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0936400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cf170 con 0x7f0930072ab0 2024-03-28T18:00:28.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0935000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09301456d0 0x7f093015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0935000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09301456d0 0x7f093015f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60354/0 (socket says 172.21.15.62:60354) 2024-03-28T18:00:28.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.709+0000 7f0935000640 1 -- 172.21.15.62:0/3542765778 learned_addr learned my addr 172.21.15.62:0/3542765778 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:28.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0927e00640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f093015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0935000640 1 -- 172.21.15.62:0/3542765778 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 msgr2=0x7f093015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:28.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0935000640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f093015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0935000640 1 -- 172.21.15.62:0/3542765778 >> v1:172.21.15.81:6789/0 conn(0x7f093006cd80 legacy=0x7f093006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0935000640 1 -- 172.21.15.62:0/3542765778 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300cf450 con 0x7f09301456d0 2024-03-28T18:00:28.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.710+0000 7f0927e00640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f093015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:28.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.711+0000 7f0935000640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09301456d0 0x7f093015f980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0920009870 tx=0x7f092002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e922a0ce9c4fe877 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f092fe00640 1 -- 172.21.15.62:0/3542765778 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920035020 con 0x7f09301456d0 2024-03-28T18:00:28.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f092fe00640 1 -- 172.21.15.62:0/3542765778 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920031640 con 0x7f09301456d0 2024-03-28T18:00:28.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f092fe00640 1 -- 172.21.15.62:0/3542765778 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092002d040 con 0x7f09301456d0 2024-03-28T18:00:28.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 -- 172.21.15.62:0/3542765778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09301456d0 msgr2=0x7f093015f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09301456d0 0x7f093015f980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0920009870 tx=0x7f092002b8f0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 -- 172.21.15.62:0/3542765778 shutdown_connections 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f093015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 --2- 172.21.15.62:0/3542765778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09301456d0 0x7f093015f980 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 -- 172.21.15.62:0/3542765778 >> 172.21.15.62:0/3542765778 conn(0x7f09300b1730 msgr2=0x7f09300b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 -- 172.21.15.62:0/3542765778 shutdown_connections 2024-03-28T18:00:28.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.712+0000 7f0936400640 1 -- 172.21.15.62:0/3542765778 wait complete. 2024-03-28T18:00:28.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.714+0000 7f0936400640 1 Processor -- start 2024-03-28T18:00:28.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.714+0000 7f0936400640 1 -- start start 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f09301449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 0x7f0930140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930071a50 con 0x7f093006cd80 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cf730 con 0x7f09301456d0 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0936400640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0930071bd0 con 0x7f0930072ab0 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.715+0000 7f0935000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60358/0 (socket says 172.21.15.62:60358) 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 -- 172.21.15.62:0/1794370801 learned_addr learned my addr 172.21.15.62:0/1794370801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935a00640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 0x7f0930140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0927e00640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f09301449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 -- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 msgr2=0x7f09301449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f09301449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 -- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 msgr2=0x7f0930140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 0x7f0930140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 -- 172.21.15.62:0/1794370801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930144f10 con 0x7f093006cd80 2024-03-28T18:00:28.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935a00640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 0x7f0930140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.716+0000 7f0935000640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f09200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.717+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920040070 con 0x7f093006cd80 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.717+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09200399f0 con 0x7f093006cd80 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.717+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920035040 con 0x7f093006cd80 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.717+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09301431b0 con 0x7f093006cd80 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.717+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0930143a50 con 0x7f093006cd80 2024-03-28T18:00:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.718+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09301435a0 con 0x7f093006cd80 2024-03-28T18:00:28.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.719+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f092004b060 con 0x7f093006cd80 2024-03-28T18:00:28.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.720+0000 7f092d600640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 0x7f090c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:28.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.720+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09200ac3c0 con 0x7f093006cd80 2024-03-28T18:00:28.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.721+0000 7f0927e00640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 0x7f090c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:28.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.722+0000 7f0927e00640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 0x7f090c06e090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0930145430 tx=0x7f0918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:28.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.725+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f092007ae10 con 0x7f093006cd80 2024-03-28T18:00:28.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.966+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f093004f4e0 con 0x7f093006cd80 2024-03-28T18:00:28.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.967+0000 7f092d600640 1 -- 172.21.15.62:0/1794370801 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4977 (secure 0 0 0) 0x7f092007a710 con 0x7f093006cd80 2024-03-28T18:00:28.967 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:28.968 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3964298394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":3964298394},{"type":"v1","addr":"172.21.15.81:6837","nonce":3964298394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/4151204969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4151204969},{"type":"v1","addr":"172.21.15.81:6836","nonce":4151204969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2711505092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2711505092},{"type":"v1","addr":"172.21.15.62:6835","nonce":2711505092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T18:00:17.037477+0000","modified":"2024-03-28T18:00:20.154562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6834/1272559486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1272559486},{"type":"v1","addr":"172.21.15.62:6834","nonce":1272559486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":2}]} 2024-03-28T18:00:28.968 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-03-28T18:00:28.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 msgr2=0x7f090c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 0x7f090c06e090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0930145430 tx=0x7f0918008040 comp rx=0 tx=0).stop 2024-03-28T18:00:28.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 msgr2=0x7f0930142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:28.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f09200046c0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 shutdown_connections 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f090c06bbd0 0x7f090c06e090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09301456d0 0x7f0930140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f0930072ab0 0x7f09301449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 --2- 172.21.15.62:0/1794370801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f093006cd80 0x7f0930142bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 >> 172.21.15.62:0/1794370801 conn(0x7f09300b1730 msgr2=0x7f093015b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 shutdown_connections 2024-03-28T18:00:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:28.971+0000 7f0936400640 1 -- 172.21.15.62:0/1794370801 wait complete. 2024-03-28T18:00:29.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T18:00:17.037477+0000', 'modified': '2024-03-28T18:00:20.154562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5208': {'gid': 5208, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6834/1272559486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 1272559486}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 1272559486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5208, 'qdb_cluster': [5208]} 2024-03-28T18:00:29.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T18:00:29.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T18:00:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.623+0000 7f4550600640 1 Processor -- start 2024-03-28T18:00:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.623+0000 7f4550600640 1 -- start start 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.624+0000 7f4550600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f454815e180 0x7f4548161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.624+0000 7f4550600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f45480b98b0 0x7f45480d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.624+0000 7f4550600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45480ce650 con 0x7f45480d0e50 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f4550600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480d12e0 con 0x7f454815e180 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f4550600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454815e560 con 0x7f45480b98b0 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454dc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f454815e180 0x7f4548161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454e600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f45480d0e50 0x7f45480b59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41476/0 (socket says 172.21.15.62:41476) 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454e600640 1 -- 172.21.15.62:0/4046481768 learned_addr learned my addr 172.21.15.62:0/4046481768 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454d200640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f45480b98b0 0x7f45480d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454dc00640 1 -- 172.21.15.62:0/4046481768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f45480b98b0 msgr2=0x7f45480d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454dc00640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f45480b98b0 0x7f45480d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.625+0000 7f454dc00640 1 -- 172.21.15.62:0/4046481768 >> v1:172.21.15.62:6789/0 conn(0x7f45480d0e50 legacy=0x7f45480b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.626+0000 7f454dc00640 1 -- 172.21.15.62:0/4046481768 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548162ae0 con 0x7f454815e180 2024-03-28T18:00:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.626+0000 7f4547e00640 1 -- 172.21.15.62:0/4046481768 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189938326 0 0) 0x7f45480ce650 con 0x7f45480d0e50 2024-03-28T18:00:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.627+0000 7f454dc00640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454815e180 0x7f4548161bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097a0 tx=0x7f453c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549e9f8fa181eb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.627+0000 7f4547e00640 1 -- 172.21.15.62:0/4046481768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c035020 con 0x7f454815e180 2024-03-28T18:00:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.627+0000 7f4547e00640 1 -- 172.21.15.62:0/4046481768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c031830 con 0x7f454815e180 2024-03-28T18:00:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.627+0000 7f4547e00640 1 -- 172.21.15.62:0/4046481768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c031c00 con 0x7f454815e180 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 -- 172.21.15.62:0/4046481768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454815e180 msgr2=0x7f4548161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454815e180 0x7f4548161bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097a0 tx=0x7f453c02b8d0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 -- 172.21.15.62:0/4046481768 shutdown_connections 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f45480b98b0 0x7f45480d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 --2- 172.21.15.62:0/4046481768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454815e180 0x7f4548161bf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 -- 172.21.15.62:0/4046481768 >> 172.21.15.62:0/4046481768 conn(0x7f4548066dd0 msgr2=0x7f45480671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 -- 172.21.15.62:0/4046481768 shutdown_connections 2024-03-28T18:00:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.628+0000 7f4550600640 1 -- 172.21.15.62:0/4046481768 wait complete. 2024-03-28T18:00:29.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.630+0000 7f4550600640 1 Processor -- start 2024-03-28T18:00:29.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.630+0000 7f4550600640 1 -- start start 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.631+0000 7f4550600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 0x7f45480c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.631+0000 7f4550600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f4550600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f454815e180 0x7f45480cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f4550600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548162800 con 0x7f45480d0e50 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f4550600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548162520 con 0x7f45480b98b0 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f4550600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480d2490 con 0x7f454815e180 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60384/0 (socket says 172.21.15.62:60384) 2024-03-28T18:00:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 -- 172.21.15.62:0/2688208125 learned_addr learned my addr 172.21.15.62:0/2688208125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454dc00640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 0x7f45480c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454e600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f454815e180 0x7f45480cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 -- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f454815e180 msgr2=0x7f45480cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f454815e180 0x7f45480cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 -- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 msgr2=0x7f45480c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 0x7f45480c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.632+0000 7f454d200640 1 -- 172.21.15.62:0/2688208125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45480cb120 con 0x7f45480d0e50 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f454dc00640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 0x7f45480c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f454d200640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f453400d830 tx=0x7f453400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f4545600640 1 -- 172.21.15.62:0/2688208125 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400b6c0 con 0x7f45480d0e50 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f4545600640 1 -- 172.21.15.62:0/2688208125 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534019070 con 0x7f45480d0e50 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45480cb400 con 0x7f45480d0e50 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f4545600640 1 -- 172.21.15.62:0/2688208125 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400ba80 con 0x7f45480d0e50 2024-03-28T18:00:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.633+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4548146c10 con 0x7f45480d0e50 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.636+0000 7f4545600640 1 -- 172.21.15.62:0/2688208125 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f453400bc20 con 0x7f45480d0e50 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.637+0000 7f45533ce740 1 Processor -- start 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.637+0000 7f45533ce740 1 -- start start 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.637+0000 7f4545600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 0x7f452806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.637+0000 7f4545600640 1 -- 172.21.15.62:0/2688208125 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f453408af70 con 0x7f45480d0e50 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f454e600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 0x7f452806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc69fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc39bf0 0x56115bc66e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bba3c90 con 0x56115bc3d750 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bb4c980 con 0x56115bc39bf0 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.638+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115bb9fd30 con 0x56115bc69bd0 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60386/0 (socket says 172.21.15.62:60386) 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 -- 172.21.15.62:0/3935804757 learned_addr learned my addr 172.21.15.62:0/3935804757 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454dc00640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc69fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454d200640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc39bf0 0x56115bc66e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 -- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 msgr2=0x56115bc69fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc69fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 -- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc39bf0 msgr2=0x56115bc66e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc39bf0 0x56115bc66e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454e600640 1 -- 172.21.15.62:0/3935804757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115bb64510 con 0x56115bc3d750 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.639+0000 7f454dc00640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.640+0000 7f454e600640 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4540005fd0 tx=0x7f4540001ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919839fcfbaf9b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.640+0000 7f453a000640 1 -- 172.21.15.62:0/3935804757 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45400197f0 con 0x56115bc3d750 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.640+0000 7f453a000640 1 -- 172.21.15.62:0/3935804757 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540011040 con 0x56115bc3d750 2024-03-28T18:00:29.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.640+0000 7f453a000640 1 -- 172.21.15.62:0/3935804757 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540019bb0 con 0x56115bc3d750 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 -- 172.21.15.62:0/3935804757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 msgr2=0x56115bc67380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f454e600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 0x7f452806e090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45400176e0 tx=0x7f454000f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4540005fd0 tx=0x7f4540001ee0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 -- 172.21.15.62:0/3935804757 shutdown_connections 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc67380 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc39bf0 0x56115bc66e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 --2- 172.21.15.62:0/3935804757 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 -- 172.21.15.62:0/3935804757 >> 172.21.15.62:0/3935804757 conn(0x56115bc3be60 msgr2=0x56115bc849e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 -- 172.21.15.62:0/3935804757 shutdown_connections 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.641+0000 7f45533ce740 1 -- 172.21.15.62:0/3935804757 wait complete. 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.642+0000 7f45533ce740 1 -- start start 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.642+0000 7f45533ce740 1 Processor -- start 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc3d750 0x56115bc5ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc5f450 0x56115bc63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bc3e900 con 0x56115bc69bd0 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bb5fdd0 con 0x56115bc3d750 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115bc3dcf0 con 0x56115bc5f450 2024-03-28T18:00:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f45533ce740 10 client.0 _ll_register_callbacks cb 0x56115b8b7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f454d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f454d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60392/0 (socket says 172.21.15.62:60392) 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f454d200640 1 -- 172.21.15.62:0/535786522 learned_addr learned my addr 172.21.15.62:0/535786522 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f454dc00640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc3d750 0x56115bc5ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.643+0000 7f454e600640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc5f450 0x56115bc63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 -- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc5f450 msgr2=0x56115bc63eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc5f450 0x56115bc63eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 -- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc3d750 msgr2=0x56115bc5ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc3d750 0x56115bc5ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 -- 172.21.15.62:0/535786522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115b8b82d0 con 0x56115bc69bd0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f454d200640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4534059050 tx=0x7f453401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614d134d9129e8f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f452ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-28T18:00:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534087060 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45340559c0 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4540011040 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4534076050 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f45533ce740 10 client.5400 fetch_fsmap learned FSMap version 17 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.644+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x56115b9826b0 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540011040 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f4534013070 con 0x56115bc69bd0 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 10 client.5400 fetch_fsmap finished waiting for FSMap version 17 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 10 client.5400 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 20 client.5400 populate_metadata read hostname 'smithi062' 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bc984f0 0x56115bc988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 4 client.5400 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:00:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 --> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56115bb45520 con 0x56115bc984f0 2024-03-28T18:00:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.645+0000 7f454dc00640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bc984f0 0x56115bc988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.646+0000 7f454dc00640 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bc984f0 0x56115bc988d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9de90a47e87cd7 server_cookie=eea3420685fe304c in_seq=0 out_seq=0 2024-03-28T18:00:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.646+0000 7f452ea00640 10 client.5400 ms_handle_connect on v2:172.21.15.62:6832/1272559486 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f452ea00640 1 -- 172.21.15.62:0/535786522 <== mds.0 v2:172.21.15.62:6832/1272559486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56115bb45520 con 0x56115bc984f0 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f452ea00640 10 client.5400 handle_command_reply: tid=0 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 client.5400 shutdown 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 msgr2=0x56115bc63970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4534059050 tx=0x7f453401c490 comp rx=0 tx=0).stop 2024-03-28T18:00:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 shutdown_connections 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bc984f0 0x56115bc988d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc5f450 0x56115bc63eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc63970 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 --2- 172.21.15.62:0/535786522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc3d750 0x56115bc5ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 >> 172.21.15.62:0/535786522 conn(0x56115bc39bf0 msgr2=0x56115bc6d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f452ea00640 0 client.5400 ms_handle_reset on v2:172.21.15.62:6832/1272559486 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 shutdown_connections 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 1 -- 172.21.15.62:0/535786522 wait complete. 2024-03-28T18:00:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.647+0000 7f45533ce740 20 client.5400 trim_cache size 0 max 16384 2024-03-28T18:00:29.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.666+0000 7f45533ce740 1 Processor -- start 2024-03-28T18:00:29.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.666+0000 7f45533ce740 1 -- start start 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc3d750 0x56115bc84cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481f7f00 con 0x56115bc69bd0 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481f7300 con 0x56115bc5f450 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45481f7900 con 0x56115bc3d750 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f454e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f454e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60402/0 (socket says 172.21.15.62:60402) 2024-03-28T18:00:29.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.667+0000 7f454e600640 1 -- 172.21.15.62:0/632066734 learned_addr learned my addr 172.21.15.62:0/632066734 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454d200640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 -- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc3d750 msgr2=0x56115bc84cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454dc00640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc3d750 0x56115bc84cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc3d750 0x56115bc84cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 -- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 msgr2=0x56115bc880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 -- 172.21.15.62:0/632066734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115bc87570 con 0x56115bc69bd0 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454d200640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:29.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.668+0000 7f454e600640 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f45400197c0 tx=0x7f4540001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a35d610c01b00dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f4539600640 1 -- 172.21.15.62:0/632066734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454002f070 con 0x56115bc69bd0 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f4539600640 1 -- 172.21.15.62:0/632066734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540007870 con 0x56115bc69bd0 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f4539600640 1 -- 172.21.15.62:0/632066734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540025040 con 0x56115bc69bd0 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 -- 172.21.15.62:0/632066734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 msgr2=0x56115bc94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f45400197c0 tx=0x7f4540001250 comp rx=0 tx=0).stop 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 -- 172.21.15.62:0/632066734 shutdown_connections 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc69bd0 0x56115bc94110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 --2- 172.21.15.62:0/632066734 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc3d750 0x56115bc84cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 -- 172.21.15.62:0/632066734 >> 172.21.15.62:0/632066734 conn(0x56115bc39bf0 msgr2=0x56115bc34ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.669+0000 7f45533ce740 1 -- 172.21.15.62:0/632066734 shutdown_connections 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.670+0000 7f45533ce740 1 -- 172.21.15.62:0/632066734 wait complete. 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.670+0000 7f45533ce740 1 -- start start 2024-03-28T18:00:29.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.670+0000 7f45533ce740 1 Processor -- start 2024-03-28T18:00:29.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bc61170 con 0x56115bc3d750 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115bc6d5a0 con 0x56115bc5f450 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.671+0000 7f45533ce740 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115bc6a0a0 con 0x56115bc69bd0 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f45533ce740 10 client.0 _ll_register_callbacks cb 0x7f453400bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60410/0 (socket says 172.21.15.62:60410) 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 -- 172.21.15.62:0/794489881 learned_addr learned my addr 172.21.15.62:0/794489881 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454e600640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454d200640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 -- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 msgr2=0x56115bc6af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc6af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 -- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 msgr2=0x56115bc6a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc6a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454dc00640 1 -- 172.21.15.62:0/794489881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115bc64120 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.672+0000 7f454e600640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f454dc00640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f453c0040e0 tx=0x7f453c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d61cb9b8976d3d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c04b030 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f453c047690 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4540007870 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f453c035040 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f45533ce740 10 client.5406 fetch_fsmap learned FSMap version 17 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x56115bc67270 con 0x56115bc3d750 2024-03-28T18:00:29.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540007870 con 0x56115bc3d750 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f453c005990 con 0x56115bc3d750 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f45533ce740 10 client.5406 fetch_fsmap finished waiting for FSMap version 17 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.673+0000 7f45533ce740 10 client.5406 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f45533ce740 20 client.5406 populate_metadata read hostname 'smithi062' 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bcf4660 0x56115bc85c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f45533ce740 4 client.5406 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 --> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56115bc6c620 con 0x56115bcf4660 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f454d200640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bcf4660 0x56115bc85c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f454d200640 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bcf4660 0x56115bc85c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b048ce1a368bcb2 server_cookie=b68d89c8b26e778c in_seq=0 out_seq=0 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.674+0000 7f452ea00640 10 client.5406 ms_handle_connect on v2:172.21.15.62:6832/1272559486 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f452ea00640 1 -- 172.21.15.62:0/794489881 <== mds.0 v2:172.21.15.62:6832/1272559486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56115bc6c620 con 0x56115bcf4660 2024-03-28T18:00:29.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f452ea00640 10 client.5406 handle_command_reply: tid=0 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 client.5406 shutdown 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 msgr2=0x56115bc63be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f453c0040e0 tx=0x7f453c004730 comp rx=0 tx=0).stop 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 shutdown_connections 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:6832/1272559486,v1:172.21.15.62:6834/1272559486] conn(0x56115bcf4660 0x56115bc85c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x56115bc69bd0 0x56115bc6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56115bc5f450 0x56115bc6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 --2- 172.21.15.62:0/794489881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56115bc3d750 0x56115bc63be0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f452ea00640 0 client.5406 ms_handle_reset on v2:172.21.15.62:6832/1272559486 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 >> 172.21.15.62:0/794489881 conn(0x56115bc39bf0 msgr2=0x56115bc880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 shutdown_connections 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 1 -- 172.21.15.62:0/794489881 wait complete. 2024-03-28T18:00:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.675+0000 7f45533ce740 20 client.5406 trim_cache size 0 max 16384 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "aedb2108-ca9d-4a87-ad07-9f4b969c6f45", 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 5208, 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 12.121738474000001, 2024-03-28T18:00:29.680 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 17, 2024-03-28T18:00:29.681 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 33, 2024-03-28T18:00:29.681 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 33, 2024-03-28T18:00:29.681 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 20.939030056 2024-03-28T18:00:29.681 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T18:00:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 msgr2=0x7f452806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 0x7f452806e090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45400176e0 tx=0x7f454000f000 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 msgr2=0x7f45480ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f453400d830 tx=0x7f453400dcf0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 shutdown_connections 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f452806bbd0 0x7f452806e090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f454815e180 0x7f45480cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45480d0e50 0x7f45480ced10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 --2- 172.21.15.62:0/2688208125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480b98b0 0x7f45480c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 >> 172.21.15.62:0/2688208125 conn(0x7f4548066dd0 msgr2=0x7f45480b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 shutdown_connections 2024-03-28T18:00:29.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.676+0000 7f4550600640 1 -- 172.21.15.62:0/2688208125 wait complete. 2024-03-28T18:00:29.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T18:00:29.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.896+0000 7f06acc00640 1 Processor -- start 2024-03-28T18:00:29.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.896+0000 7f06acc00640 1 -- start start 2024-03-28T18:00:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06acc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06a806f4a0 0x7f06a8072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06acc00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06a80b45e0 0x7f06a80ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06acc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a8071c50 con 0x7f06a80b4200 2024-03-28T18:00:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06acc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a806fa80 con 0x7f06a806f4a0 2024-03-28T18:00:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06acc00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a8071ef0 con 0x7f06a80b45e0 2024-03-28T18:00:29.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f069d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06a806f4a0 0x7f06a8072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06a5e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f06a80b4200 0x7f06a815efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41486/0 (socket says 172.21.15.62:41486) 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.898+0000 7f06a5e00640 1 -- 172.21.15.62:0/4047113926 learned_addr learned my addr 172.21.15.62:0/4047113926 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f06a6800640 1 --2- 172.21.15.62:0/4047113926 >> v2:172.21.15.81:3300/0 conn(0x7f06a80b45e0 0x7f06a80ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f069d400640 1 -- 172.21.15.62:0/4047113926 >> v2:172.21.15.81:3300/0 conn(0x7f06a80b45e0 msgr2=0x7f06a80ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f069d400640 1 --2- 172.21.15.62:0/4047113926 >> v2:172.21.15.81:3300/0 conn(0x7f06a80b45e0 0x7f06a80ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f069d400640 1 -- 172.21.15.62:0/4047113926 >> v1:172.21.15.62:6789/0 conn(0x7f06a80b4200 legacy=0x7f06a815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f069d400640 1 -- 172.21.15.62:0/4047113926 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80721d0 con 0x7f06a806f4a0 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.899+0000 7f06a5400640 1 -- 172.21.15.62:0/4047113926 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878095913 0 0) 0x7f06a8071c50 con 0x7f06a80b4200 2024-03-28T18:00:29.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.900+0000 7f069d400640 1 --2- 172.21.15.62:0/4047113926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a8072cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b1d50 tx=0x7f069002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f559c1beda1a8c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.900+0000 7f06a5400640 1 -- 172.21.15.62:0/4047113926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900315a0 con 0x7f06a806f4a0 2024-03-28T18:00:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.901+0000 7f06a5400640 1 -- 172.21.15.62:0/4047113926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690037070 con 0x7f06a806f4a0 2024-03-28T18:00:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.901+0000 7f06a5400640 1 -- 172.21.15.62:0/4047113926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690031960 con 0x7f06a806f4a0 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.901+0000 7f06acc00640 1 -- 172.21.15.62:0/4047113926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 msgr2=0x7f06a8072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.901+0000 7f06acc00640 1 --2- 172.21.15.62:0/4047113926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a8072cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b1d50 tx=0x7f069002b840 comp rx=0 tx=0).stop 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 -- 172.21.15.62:0/4047113926 shutdown_connections 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 --2- 172.21.15.62:0/4047113926 >> v2:172.21.15.81:3300/0 conn(0x7f06a80b45e0 0x7f06a80ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 --2- 172.21.15.62:0/4047113926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a8072cb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 -- 172.21.15.62:0/4047113926 >> 172.21.15.62:0/4047113926 conn(0x7f06a806d0e0 msgr2=0x7f06a806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 -- 172.21.15.62:0/4047113926 shutdown_connections 2024-03-28T18:00:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.902+0000 7f06acc00640 1 -- 172.21.15.62:0/4047113926 wait complete. 2024-03-28T18:00:29.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.904+0000 7f06acc00640 1 Processor -- start 2024-03-28T18:00:29.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.904+0000 7f06acc00640 1 -- start start 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.905+0000 7f06acc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06acc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06a80b4200 0x7f06a8136740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06acc00640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06a80b45e0 0x7f06a8146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06acc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80b4a50 con 0x7f06a806f4a0 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06acc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8072790 con 0x7f06a80b4200 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06acc00640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8147d90 con 0x7f06a80b45e0 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60422/0 (socket says 172.21.15.62:60422) 2024-03-28T18:00:29.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 -- 172.21.15.62:0/1462771443 learned_addr learned my addr 172.21.15.62:0/1462771443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f069d400640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06a80b4200 0x7f06a8136740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a6800640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06a80b45e0 0x7f06a8146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 -- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06a80b45e0 msgr2=0x7f06a8146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06a80b45e0 0x7f06a8146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 -- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06a80b4200 msgr2=0x7f06a8136740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.906+0000 7f06a5e00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06a80b4200 0x7f06a8136740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f06a5e00640 1 -- 172.21.15.62:0/1462771443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a81f5c30 con 0x7f06a806f4a0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f06a5e00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f069800c4d0 tx=0x7f069800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698018070 con 0x7f06a806f4a0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698007500 con 0x7f06a806f4a0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698004a20 con 0x7f06a806f4a0 2024-03-28T18:00:29.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a81f5f10 con 0x7f06a806f4a0 2024-03-28T18:00:29.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.907+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a81f64d0 con 0x7f06a806f4a0 2024-03-28T18:00:29.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.909+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a80680c0 con 0x7f06a806f4a0 2024-03-28T18:00:29.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.910+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0698004bc0 con 0x7f06a806f4a0 2024-03-28T18:00:29.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.912+0000 7f069ea00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 0x7f068406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:29.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.912+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0698089db0 con 0x7f06a806f4a0 2024-03-28T18:00:29.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.912+0000 7f069d400640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 0x7f068406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:29.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.913+0000 7f069d400640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 0x7f068406e090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0690036000 tx=0x7f0690008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:29.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:29.916+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f06980588a0 con 0x7f06a806f4a0 2024-03-28T18:00:30.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.160+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06a80cc4e0 con 0x7f06a806f4a0 2024-03-28T18:00:30.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.160+0000 7f069ea00640 1 -- 172.21.15.62:0/1462771443 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06980581a0 con 0x7f06a806f4a0 2024-03-28T18:00:30.161 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T18:00:30.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 msgr2=0x7f068406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:30.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 0x7f068406e090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0690036000 tx=0x7f0690008030 comp rx=0 tx=0).stop 2024-03-28T18:00:30.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 msgr2=0x7f06a81f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:30.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f069800c4d0 tx=0x7f069800c990 comp rx=0 tx=0).stop 2024-03-28T18:00:30.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 shutdown_connections 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f068406bbd0 0x7f068406e090 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f06a80b45e0 0x7f06a8146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06a80b4200 0x7f06a8136740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 --2- 172.21.15.62:0/1462771443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a806f4a0 0x7f06a81f6cd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 >> 172.21.15.62:0/1462771443 conn(0x7f06a806d0e0 msgr2=0x7f06a80adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 shutdown_connections 2024-03-28T18:00:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:30.163+0000 7f06acc00640 1 -- 172.21.15.62:0/1462771443 wait complete. 2024-03-28T18:00:30.182 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T18:00:30.182 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T18:00:30.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-03-28T18:00:30.182 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T18:00:30.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:30.183 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T18:00:30.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T18:00:30.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T18:00:30.183 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T18:00:30.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.183 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-03-28T18:00:30.232 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T18:00:30.232 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T18:00:30.232 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:00:30.233 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42495sec preferred_lft 42495sec 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:90:41:79:94:ec brd ff:ff:ff:ff:ff:ff 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2c96:10ff:fe48:6677/64 scope link 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:90:41:79:94:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::890:41ff:fe79:94ec/64 scope link 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.234 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T18:00:30.235 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 72:ef:a4:09:4f:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:30.235 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::70ef:a4ff:fe09:4f52/64 scope link 2024-03-28T18:00:30.235 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:00:30.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.236 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T18:00:30.322 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-03-28T18:00:30.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.327 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-03-28T18:00:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-03-28T18:00:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T18:00:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T18:00:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T18:00:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T18:00:30.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.380 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-03-28T18:00:30.380 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-03-28T18:00:30.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T18:00:30.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T18:00:30.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T18:00:30.381 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-03-28T18:00:30.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.570 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-03-28T18:00:30.623 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T18:00:30.623 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T18:00:30.624 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T18:00:30.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-03-28T18:00:30.726 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-28T18:00:30.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T18:00:30.726 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:30.776 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T18:00:30.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T18:00:30.777 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:30.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T18:00:30.920 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T18:00:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:00:30.978 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:30.979 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:30.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:30.980 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T18:00:31.029 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-28T18:00:31.030 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-28T18:00:31.035 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T18:00:31.109 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:00:31.153 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T18:00:31.153 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T18:00:31.154 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T18:00:31.155 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:31.156 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:31.156 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:31.156 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:31.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:31.157 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T18:00:31.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:2024-03-28T18:00:31.265+0000 7f4af540d480 -1 init, newargv = 0x55592b7ebbd0 newargc=15 2024-03-28T18:00:31.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[50812]: starting ceph client 2024-03-28T18:00:31.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[50812]: starting fuse 2024-03-28T18:00:32.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-03-28T18:00:32.446 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:00:32.497 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-03-28T18:00:32.497 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-03-28T18:00:32.497 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-03-28T18:00:32.497 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-03-28T18:00:32.498 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-03-28T18:00:32.499 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-03-28T18:00:32.499 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-03-28T18:00:32.499 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-03-28T18:00:32.499 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-03-28T18:00:32.499 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:00:32.503 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:00:32.503 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-0 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-1 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-2 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph/osd/ceph-3 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T18:00:32.504 INFO:teuthology.orchestra.run.smithi062.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:32.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:32.505 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-03-28T18:00:32.555 INFO:teuthology.orchestra.run.smithi062.stdout:42 2024-03-28T18:00:32.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-03-28T18:00:32.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo stdin-killer -- python3 -c ' 2024-03-28T18:00:32.555 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-03-28T18:00:32.555 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-03-28T18:00:32.556 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.0")) 2024-03-28T18:00:32.557 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-03-28T18:00:32.733 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.0.50812.asok 2024-03-28T18:00:32.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T18:00:32.751 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50812.asok 2024-03-28T18:00:32.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:32.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50812.asok status 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "0", 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-03-28T18:00:32.985 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "50812", 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 5424, 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 5424 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T18:00:32.986 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3469833924 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3469833924 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.5424 192.168.0.1:0/3469833924", 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.1:0/3469833924", 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 17, 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 33, 2024-03-28T18:00:32.987 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-03-28T18:00:32.988 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-03-28T18:00:32.988 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-03-28T18:00:32.988 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-28T18:00:33.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:33.000 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-03-28T18:00:33.000 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:33.050 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-03-28T18:00:33.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:33.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:33.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T18:00:33.144 DEBUG:teuthology.orchestra.run.smithi062:> mktemp -d --suffix=xfstests-dev 2024-03-28T18:00:33.196 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.7I1MPLSTiQxfstests-dev 2024-03-28T18:00:33.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:33.197 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.7I1MPLSTiQxfstests-dev) 2024-03-28T18:00:33.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:33.291 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.7I1MPLSTiQxfstests-dev) 2024-03-28T18:00:33.349 INFO:teuthology.orchestra.run.smithi062.stderr:Cloning into '/tmp/tmp.7I1MPLSTiQxfstests-dev'... 2024-03-28T18:00:34.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:34.511 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-03-28T18:00:34.573 DEBUG:teuthology.orchestra.run.smithi062:> mktemp -d --suffix=test 2024-03-28T18:00:34.622 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.sy6xOdcDqftest 2024-03-28T18:00:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:00:34.622 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-03-28T18:00:34.673 DEBUG:teuthology.orchestra.run.smithi062:> mktemp -d --suffix=scratch 2024-03-28T18:00:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.NdNTLS4TFuscratch 2024-03-28T18:00:34.716 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-03-28T18:00:34.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-03-28T18:00:35.617 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:04:33 ago on Thu 28 Mar 2024 05:56:02 PM UTC. 2024-03-28T18:00:35.722 INFO:teuthology.orchestra.run.smithi062.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-03-28T18:00:35.723 INFO:teuthology.orchestra.run.smithi062.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T18:00:35.724 INFO:teuthology.orchestra.run.smithi062.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-03-28T18:00:35.725 INFO:teuthology.orchestra.run.smithi062.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-03-28T18:00:35.726 INFO:teuthology.orchestra.run.smithi062.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-03-28T18:00:35.728 INFO:teuthology.orchestra.run.smithi062.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-03-28T18:00:35.730 INFO:teuthology.orchestra.run.smithi062.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-03-28T18:00:35.731 INFO:teuthology.orchestra.run.smithi062.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-03-28T18:00:35.733 INFO:teuthology.orchestra.run.smithi062.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-03-28T18:00:35.736 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-03-28T18:00:35.738 INFO:teuthology.orchestra.run.smithi062.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-03-28T18:00:35.740 INFO:teuthology.orchestra.run.smithi062.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-03-28T18:00:35.741 INFO:teuthology.orchestra.run.smithi062.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-03-28T18:00:35.742 INFO:teuthology.orchestra.run.smithi062.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-03-28T18:00:35.743 INFO:teuthology.orchestra.run.smithi062.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-03-28T18:00:35.744 INFO:teuthology.orchestra.run.smithi062.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-03-28T18:00:35.745 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-03-28T18:00:35.746 INFO:teuthology.orchestra.run.smithi062.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-03-28T18:00:35.747 INFO:teuthology.orchestra.run.smithi062.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-03-28T18:00:35.750 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-28T18:00:35.825 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-03-28T18:00:35.829 INFO:teuthology.orchestra.run.smithi062.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-devel x86_64 5.14.0-432.el9 appstream 16 M 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: cpp x86_64 11.4.1-3.el9 appstream 11 M 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper x86_64 9:1.02.197-2.el9 baseos 141 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 baseos 34 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 32 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 baseos 179 k 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: e2fsprogs x86_64 1.46.5-5.el9 baseos 1.0 M 2024-03-28T18:00:35.830 INFO:teuthology.orchestra.run.smithi062.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 baseos 222 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: gcc x86_64 11.4.1-3.el9 appstream 32 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: gcc-c++ x86_64 11.4.1-3.el9 appstream 13 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 appstream 47 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: git x86_64 2.43.0-1.el9 appstream 51 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: git-core x86_64 2.43.0-1.el9 appstream 4.4 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: git-core-doc noarch 2.43.0-1.el9 appstream 2.9 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err x86_64 1.46.5-5.el9 baseos 26 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libgcc x86_64 11.4.1-3.el9 baseos 92 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libgomp x86_64 11.4.1-3.el9 baseos 268 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libss x86_64 1.46.5-5.el9 baseos 32 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libstdc++ x86_64 11.4.1-3.el9 baseos 743 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: libstdc++-devel x86_64 11.4.1-3.el9 appstream 2.4 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: make x86_64 1:4.3-8.el9 baseos 536 k 2024-03-28T18:00:35.831 INFO:teuthology.orchestra.run.smithi062.stdout: perl-Git noarch 2.43.0-1.el9 appstream 38 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-28T18:00:35.832 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:Install 17 Packages 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 28 Packages 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 103 M 2024-03-28T18:00:35.833 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-03-28T18:00:37.148 INFO:teuthology.orchestra.run.smithi062.stdout:(1/45): fio-engine-http-3.35-1.el9.x86_64.rpm 68 kB/s | 17 kB 00:00 2024-03-28T18:00:37.173 INFO:teuthology.orchestra.run.smithi062.stdout:(2/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 53 kB/s | 14 kB 00:00 2024-03-28T18:00:37.232 INFO:teuthology.orchestra.run.smithi062.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 168 kB/s | 13 kB 00:00 2024-03-28T18:00:37.333 INFO:teuthology.orchestra.run.smithi062.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 92 kB/s | 15 kB 00:00 2024-03-28T18:00:37.358 INFO:teuthology.orchestra.run.smithi062.stdout:(5/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 121 kB/s | 15 kB 00:00 2024-03-28T18:00:37.425 INFO:teuthology.orchestra.run.smithi062.stdout:(6/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 215 kB/s | 20 kB 00:00 2024-03-28T18:00:37.592 INFO:teuthology.orchestra.run.smithi062.stdout:(7/45): fio-3.35-1.el9.x86_64.rpm 9.0 MB/s | 6.2 MB 00:00 2024-03-28T18:00:37.617 INFO:teuthology.orchestra.run.smithi062.stdout:(8/45): libcap-devel-2.48-9.el9.x86_64.rpm 136 kB/s | 33 kB 00:00 2024-03-28T18:00:37.659 INFO:teuthology.orchestra.run.smithi062.stdout:(9/45): sqlite-3.34.1-7.el9.x86_64.rpm 3.3 MB/s | 747 kB 00:00 2024-03-28T18:00:37.701 INFO:teuthology.orchestra.run.smithi062.stdout:(10/45): udftools-2.2-5.el9.x86_64.rpm 1.9 MB/s | 144 kB 00:00 2024-03-28T18:00:37.726 INFO:teuthology.orchestra.run.smithi062.stdout:(11/45): rmt-1.6-6.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-03-28T18:00:37.918 INFO:teuthology.orchestra.run.smithi062.stdout:(12/45): gdbm-1.19-4.el9.x86_64.rpm 695 kB/s | 132 kB 00:00 2024-03-28T18:00:37.977 INFO:teuthology.orchestra.run.smithi062.stdout:(13/45): gdbm-devel-1.19-4.el9.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2024-03-28T18:00:38.469 INFO:teuthology.orchestra.run.smithi062.stdout:(14/45): dump-0.4-0.51.b47.el9.x86_64.rpm 179 kB/s | 88 kB 00:00 2024-03-28T18:00:38.611 INFO:teuthology.orchestra.run.smithi062.stdout:(15/45): indent-2.2.13-5.el9.x86_64.rpm 1.2 MB/s | 179 kB 00:00 2024-03-28T18:00:38.646 INFO:teuthology.orchestra.run.smithi062.stdout:(16/45): python-unversioned-command-3.9.18-3.el 256 kB/s | 8.3 kB 00:00 2024-03-28T18:00:38.686 INFO:teuthology.orchestra.run.smithi062.stdout:(17/45): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 204 kB 00:00 2024-03-28T18:00:38.737 INFO:teuthology.orchestra.run.smithi062.stdout:(18/45): libnbd-1.18.1-3.el9.x86_64.rpm 165 kB/s | 166 kB 00:01 2024-03-28T18:00:38.762 INFO:teuthology.orchestra.run.smithi062.stdout:(19/45): python3-3.9.18-3.el9.x86_64.rpm 337 kB/s | 25 kB 00:00 2024-03-28T18:00:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:(20/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 6.7 MB/s | 619 kB 00:00 2024-03-28T18:00:38.996 INFO:teuthology.orchestra.run.smithi062.stdout:(21/45): python3-libs-3.9.18-3.el9.x86_64.rpm 28 MB/s | 7.3 MB 00:00 2024-03-28T18:00:39.080 INFO:teuthology.orchestra.run.smithi062.stdout:(22/45): device-mapper-1.02.197-2.el9.x86_64.rp 687 kB/s | 141 kB 00:00 2024-03-28T18:00:39.105 INFO:teuthology.orchestra.run.smithi062.stdout:(23/45): device-mapper-event-1.02.197-2.el9.x86 318 kB/s | 34 kB 00:00 2024-03-28T18:00:39.172 INFO:teuthology.orchestra.run.smithi062.stdout:(24/45): device-mapper-event-libs-1.02.197-2.el 355 kB/s | 32 kB 00:00 2024-03-28T18:00:39.197 INFO:teuthology.orchestra.run.smithi062.stdout:(25/45): device-mapper-libs-1.02.197-2.el9.x86_ 1.9 MB/s | 179 kB 00:00 2024-03-28T18:00:39.298 INFO:teuthology.orchestra.run.smithi062.stdout:(26/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 2.2 MB/s | 222 kB 00:00 2024-03-28T18:00:39.356 INFO:teuthology.orchestra.run.smithi062.stdout:(27/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-03-28T18:00:39.399 INFO:teuthology.orchestra.run.smithi062.stdout:(28/45): libcom_err-1.46.5-5.el9.x86_64.rpm 265 kB/s | 26 kB 00:00 2024-03-28T18:00:39.482 INFO:teuthology.orchestra.run.smithi062.stdout:(29/45): libgcc-11.4.1-3.el9.x86_64.rpm 743 kB/s | 92 kB 00:00 2024-03-28T18:00:39.515 INFO:teuthology.orchestra.run.smithi062.stdout:(30/45): libgomp-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 268 kB 00:00 2024-03-28T18:00:39.566 INFO:teuthology.orchestra.run.smithi062.stdout:(31/45): libss-1.46.5-5.el9.x86_64.rpm 389 kB/s | 32 kB 00:00 2024-03-28T18:00:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:(32/45): libstdc++-11.4.1-3.el9.x86_64.rpm 5.1 MB/s | 743 kB 00:00 2024-03-28T18:00:39.866 INFO:teuthology.orchestra.run.smithi062.stdout:(33/45): lvm2-2.03.23-2.el9.x86_64.rpm 5.1 MB/s | 1.5 MB 00:00 2024-03-28T18:00:39.992 INFO:teuthology.orchestra.run.smithi062.stdout:(34/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-03-28T18:00:40.167 INFO:teuthology.orchestra.run.smithi062.stdout:(35/45): make-4.3-8.el9.x86_64.rpm 1.7 MB/s | 536 kB 00:00 2024-03-28T18:00:44.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:00:44.426 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:00:54.334 INFO:teuthology.orchestra.run.smithi062.stdout:(36/45): cpp-11.4.1-3.el9.x86_64.rpm 759 kB/s | 11 MB 00:14 2024-03-28T18:00:57.901 INFO:teuthology.orchestra.run.smithi062.stdout:(37/45): kernel-devel-5.14.0-432.el9.x86_64.rpm 790 kB/s | 16 MB 00:20 2024-03-28T18:00:58.584 INFO:teuthology.orchestra.run.smithi062.stdout:(38/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 69 kB/s | 47 kB 00:00 2024-03-28T18:00:59.284 INFO:teuthology.orchestra.run.smithi062.stdout:(39/45): git-2.43.0-1.el9.x86_64.rpm 73 kB/s | 51 kB 00:00 2024-03-28T18:01:05.409 INFO:teuthology.orchestra.run.smithi062.stdout:(40/45): git-core-2.43.0-1.el9.x86_64.rpm 741 kB/s | 4.4 MB 00:06 2024-03-28T18:01:10.333 INFO:teuthology.orchestra.run.smithi062.stdout:(41/45): git-core-doc-2.43.0-1.el9.noarch.rpm 608 kB/s | 2.9 MB 00:04 2024-03-28T18:01:12.190 INFO:teuthology.orchestra.run.smithi062.stdout:(42/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 733 kB/s | 13 MB 00:17 2024-03-28T18:01:12.865 INFO:teuthology.orchestra.run.smithi062.stdout:(43/45): perl-Git-2.43.0-1.el9.noarch.rpm 56 kB/s | 38 kB 00:00 2024-03-28T18:01:14.040 INFO:teuthology.orchestra.run.smithi062.stdout:(44/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 668 kB/s | 2.4 MB 00:03 2024-03-28T18:01:14.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:01:14.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:01:21.399 INFO:teuthology.orchestra.run.smithi062.stdout:(45/45): gcc-11.4.1-3.el9.x86_64.rpm 801 kB/s | 32 MB 00:41 2024-03-28T18:01:21.405 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-03-28T18:01:21.405 INFO:teuthology.orchestra.run.smithi062.stdout:Total 2.3 MB/s | 103 MB 00:45 2024-03-28T18:01:23.195 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-03-28T18:01:23.634 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-03-28T18:01:23.635 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-03-28T18:01:25.259 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-03-28T18:01:25.260 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-03-28T18:01:28.775 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-03-28T18:01:28.824 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-28T18:01:28.907 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-28T18:01:29.004 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-03-28T18:01:29.096 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-03-28T18:01:29.182 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-03-28T18:01:29.270 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-03-28T18:01:29.305 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-03-28T18:01:29.701 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-03-28T18:01:29.952 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 8/73 2024-03-28T18:01:30.013 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : make-1:4.3-8.el9.x86_64 9/73 2024-03-28T18:01:31.743 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 10/73 2024-03-28T18:01:31.930 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 11/73 2024-03-28T18:01:31.964 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 12/73 2024-03-28T18:01:32.963 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 13/73 2024-03-28T18:01:33.088 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-03-28T18:01:33.154 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-03-28T18:01:33.225 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-03-28T18:01:33.260 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-28T18:01:33.696 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-28T18:01:34.420 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-03-28T18:01:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-03-28T18:01:34.898 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-03-28T18:01:35.124 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 21/73 2024-03-28T18:01:36.454 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 22/73 2024-03-28T18:01:36.672 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-03-28T18:01:36.801 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-03-28T18:01:36.884 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-03-28T18:01:36.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 25/73 2024-03-28T18:01:36.952 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 26/73 2024-03-28T18:01:36.986 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 27/73 2024-03-28T18:01:37.016 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 28/73 2024-03-28T18:01:37.074 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 29/73 2024-03-28T18:01:37.113 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 30/73 2024-03-28T18:01:37.380 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 31/73 2024-03-28T18:01:37.433 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fio-3.35-1.el9.x86_64 32/73 2024-03-28T18:01:37.473 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rmt-2:1.6-6.el9.x86_64 33/73 2024-03-28T18:01:37.511 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-03-28T18:01:44.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:01:44.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:01:50.193 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-03-28T18:01:50.364 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-devel-5.14.0-432.el9.x86_64 36/73 2024-03-28T18:02:14.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:02:14.868 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:02:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-devel-5.14.0-432.el9.x86_64 36/73 2024-03-28T18:02:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-03-28T18:02:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-28T18:02:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-28T18:02:43.627 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-03-28T18:02:43.627 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-28T18:02:43.627 INFO:teuthology.orchestra.run.smithi062.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-03-28T18:02:43.627 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:02:44.205 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-28T18:02:44.267 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-03-28T18:02:44.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-03-28T18:02:44.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-03-28T18:02:44.460 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-03-28T18:02:44.495 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-03-28T18:02:44.533 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-03-28T18:02:44.578 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 47/73 2024-03-28T18:02:44.625 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 48/73 2024-03-28T18:02:44.672 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 49/73 2024-03-28T18:02:44.764 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-03-28T18:02:44.789 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-03-28T18:02:44.790 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-03-28T18:02:44.901 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-03-28T18:02:44.951 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 53/73 2024-03-28T18:02:44.985 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-28T18:02:44.985 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-28T18:02:45.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:02:45.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:02:45.126 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-28T18:02:45.212 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 55/73 2024-03-28T18:02:45.250 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 56/73 2024-03-28T18:02:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 57/73 2024-03-28T18:02:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 58/73 2024-03-28T18:02:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-03-28T18:02:45.389 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-03-28T18:02:45.463 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 60/73 2024-03-28T18:02:45.503 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 61/73 2024-03-28T18:02:45.537 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-03-28T18:02:45.570 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-03-28T18:02:45.605 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-03-28T18:02:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-03-28T18:02:45.850 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-03-28T18:02:45.891 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-03-28T18:02:45.925 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-03-28T18:02:45.925 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-28T18:02:46.067 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-28T18:02:46.113 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 70/73 2024-03-28T18:02:46.170 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 71/73 2024-03-28T18:02:46.213 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 72/73 2024-03-28T18:02:46.348 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : make-1:4.3-7.el9.x86_64 73/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: make-1:4.3-7.el9.x86_64 73/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-03-28T18:02:48.479 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 8/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 9/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : udftools-2.2-5.el9.x86_64 10/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 11/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-devel-5.14.0-432.el9.x86_64 12/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 13/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 18/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 19/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 20/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 21/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-3.9.18-3.el9.x86_64 22/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-3.9.17-2.el9.x86_64 23/73 2024-03-28T18:02:48.480 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 24/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 25/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 26/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 27/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 28/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 29/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 30/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 31/73 2024-03-28T18:02:48.481 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 32/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 33/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 34/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 35/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 36/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 37/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 38/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 39/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 40/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 41/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 42/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 43/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 44/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 45/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libss-1.46.5-5.el9.x86_64 46/73 2024-03-28T18:02:48.485 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libss-1.46.5-3.el9.x86_64 47/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 48/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 49/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 50/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 51/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 52/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : make-1:4.3-8.el9.x86_64 54/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : make-1:4.3-7.el9.x86_64 55/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 56/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 57/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 58/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 59/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 60/73 2024-03-28T18:02:48.486 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 61/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 62/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 63/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-2.43.0-1.el9.x86_64 64/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-2.39.3-1.el9.x86_64 65/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 66/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 67/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 68/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 69/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 70/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 71/73 2024-03-28T18:02:48.487 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 72/73 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 73/73 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: cpp-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-03-28T18:02:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: gcc-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: git-2.43.0-1.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: git-core-2.43.0-1.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: libgcc-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: libgomp-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: libss-1.46.5-5.el9.x86_64 2024-03-28T18:02:49.809 INFO:teuthology.orchestra.run.smithi062.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: make-1:4.3-8.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: perl-Git-2.43.0-1.el9.noarch 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: python3-3.9.18-3.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-03-28T18:02:49.810 INFO:teuthology.orchestra.run.smithi062.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-432.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: udftools-2.2-5.el9.x86_64 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:02:49.811 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-03-28T18:02:50.425 DEBUG:teuthology.orchestra.run.smithi062:> sudo useradd -m fsgqa 2024-03-28T18:02:50.514 INFO:teuthology.orchestra.run.smithi062.stderr:useradd: user 'fsgqa' already exists 2024-03-28T18:02:50.517 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-28T18:02:50.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo groupadd fsgqa 2024-03-28T18:02:50.584 INFO:teuthology.orchestra.run.smithi062.stderr:groupadd: group 'fsgqa' already exists 2024-03-28T18:02:50.588 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-28T18:02:50.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo useradd fsgqa2 2024-03-28T18:02:51.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo useradd 123456-fsgqa 2024-03-28T18:02:52.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-03-28T18:02:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.262+0000 7fa179600640 1 Processor -- start 2024-03-28T18:02:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.262+0000 7fa179600640 1 -- start start 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.265+0000 7fa179600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa17415dc00 0x7fa1740cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.265+0000 7fa179600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1740cf190 con 0x7fa174071630 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.265+0000 7fa179600640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa174070ae0 con 0x7fa1740b4370 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.265+0000 7fa179600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa174070dc0 con 0x7fa17415dc00 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.265+0000 7fa172a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa17415dc00 0x7fa1740cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa17415dc00 0x7fa1740cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33556/0 (socket says 172.21.15.62:33556) 2024-03-28T18:02:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172a00640 1 -- 172.21.15.62:0/3632305031 learned_addr learned my addr 172.21.15.62:0/3632305031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:02:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172a00640 1 -- 172.21.15.62:0/3632305031 >> v1:172.21.15.81:6789/0 conn(0x7fa1740b4370 legacy=0x7fa17406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:02:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172000640 1 -- 172.21.15.62:0/3632305031 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307368184 0 0) 0x7fa1740cf190 con 0x7fa174071630 2024-03-28T18:02:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172a00640 1 -- 172.21.15.62:0/3632305031 >> v1:172.21.15.62:6789/0 conn(0x7fa174071630 legacy=0x7fa1740ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.266+0000 7fa172a00640 1 -- 172.21.15.62:0/3632305031 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740cf430 con 0x7fa17415dc00 2024-03-28T18:02:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa172a00640 1 --2- 172.21.15.62:0/3632305031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17415dc00 0x7fa1740cc4e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa16400aa10 tx=0x7fa16402cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b9c2dffe6c5762 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa172000640 1 -- 172.21.15.62:0/3632305031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16403a070 con 0x7fa17415dc00 2024-03-28T18:02:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa172000640 1 -- 172.21.15.62:0/3632305031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16402dca0 con 0x7fa17415dc00 2024-03-28T18:02:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa172000640 1 -- 172.21.15.62:0/3632305031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400b040 con 0x7fa17415dc00 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa179600640 1 -- 172.21.15.62:0/3632305031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17415dc00 msgr2=0x7fa1740cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.269+0000 7fa179600640 1 --2- 172.21.15.62:0/3632305031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17415dc00 0x7fa1740cc4e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa16400aa10 tx=0x7fa16402cef0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.270+0000 7fa179600640 1 -- 172.21.15.62:0/3632305031 shutdown_connections 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.270+0000 7fa179600640 1 --2- 172.21.15.62:0/3632305031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17415dc00 0x7fa1740cc4e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.270+0000 7fa179600640 1 -- 172.21.15.62:0/3632305031 >> 172.21.15.62:0/3632305031 conn(0x7fa1740b1650 msgr2=0x7fa1740b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.270+0000 7fa179600640 1 -- 172.21.15.62:0/3632305031 shutdown_connections 2024-03-28T18:02:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.270+0000 7fa179600640 1 -- 172.21.15.62:0/3632305031 wait complete. 2024-03-28T18:02:52.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.272+0000 7fa179600640 1 Processor -- start 2024-03-28T18:02:52.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.273+0000 7fa179600640 1 -- start start 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1740b4370 0x7fa1740cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa17415dc00 0x7fa1740cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740aee30 con 0x7fa174071630 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740aecb0 con 0x7fa1740b4370 2024-03-28T18:02:52.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa179600640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17406da00 con 0x7fa17415dc00 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa172a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa172a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33560/0 (socket says 172.21.15.62:33560) 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.274+0000 7fa172a00640 1 -- 172.21.15.62:0/4279031237 learned_addr learned my addr 172.21.15.62:0/4279031237 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa173400640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa17415dc00 0x7fa1740cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa16be00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1740b4370 0x7fa1740cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 -- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa17415dc00 msgr2=0x7fa1740cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa17415dc00 0x7fa1740cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 -- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1740b4370 msgr2=0x7fa1740cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1740b4370 0x7fa1740cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 -- 172.21.15.62:0/4279031237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174143ba0 con 0x7fa174071630 2024-03-28T18:02:52.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.275+0000 7fa172a00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa164038040 tx=0x7fa1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.276+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164044070 con 0x7fa174071630 2024-03-28T18:02:52.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.276+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16402de50 con 0x7fa174071630 2024-03-28T18:02:52.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.276+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16403a070 con 0x7fa174071630 2024-03-28T18:02:52.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.276+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa174143e80 con 0x7fa174071630 2024-03-28T18:02:52.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.276+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa174144440 con 0x7fa174071630 2024-03-28T18:02:52.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.278+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1740680c0 con 0x7fa174071630 2024-03-28T18:02:52.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.279+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa164047070 con 0x7fa174071630 2024-03-28T18:02:52.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.279+0000 7fa16aa00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 0x7fa15006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.280+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1640ac0c0 con 0x7fa174071630 2024-03-28T18:02:52.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.280+0000 7fa16be00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 0x7fa15006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.281+0000 7fa16be00640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 0x7fa15006e0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1740cd080 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.285+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7fa16407ac90 con 0x7fa174071630 2024-03-28T18:02:52.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.574+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa1740b3430 con 0x7fa174071630 2024-03-28T18:02:52.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.574+0000 7fa16aa00640 1 -- 172.21.15.62:0/4279031237 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1277 (secure 0 0 0) 0x7fa16407a590 con 0x7fa174071630 2024-03-28T18:02:52.575 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:02:52.575 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"aedb2108-ca9d-4a87-ad07-9f4b969c6f45","modified":"2024-03-28T17:57:46.267099Z","created":"2024-03-28T17:57:46.267099Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3301","nonce":0},{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-28T18:02:52.575 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-03-28T18:02:52.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 msgr2=0x7fa15006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 0x7fa15006e0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1740cd080 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 msgr2=0x7fa174142240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa164038040 tx=0x7fa1640046c0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 shutdown_connections 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7fa15006bc20 0x7fa15006e0e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fa17415dc00 0x7fa1740cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1740b4370 0x7fa1740cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 --2- 172.21.15.62:0/4279031237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa174071630 0x7fa174142240 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 >> 172.21.15.62:0/4279031237 conn(0x7fa1740b1650 msgr2=0x7fa1740cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:52.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 shutdown_connections 2024-03-28T18:02:52.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.577+0000 7fa179600640 1 -- 172.21.15.62:0/4279031237 wait complete. 2024-03-28T18:02:52.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-03-28T18:02:52.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.809+0000 7f2703800640 1 Processor -- start 2024-03-28T18:02:52.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.809+0000 7f2703800640 1 -- start start 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.811+0000 7f2703800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26fc0719b0 0x7f26fc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.811+0000 7f2703800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.811+0000 7f2703800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc0b30d0 con 0x7f26fc0ae370 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.811+0000 7f2703800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0b33b0 con 0x7f26fc0719b0 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.811+0000 7f2703800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b3690 con 0x7f26fc071d90 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f26fbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f26fc0ae370 0x7f26fc0ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37340/0 (socket says 172.21.15.62:37340) 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f2700e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26fc0719b0 0x7f26fc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f26fbe00640 1 -- 172.21.15.62:0/2901130738 learned_addr learned my addr 172.21.15.62:0/2901130738 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-03-28T18:02:52.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f2701800640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f2700e00640 1 -- 172.21.15.62:0/2901130738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 msgr2=0x7f26fc072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:02:52.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f2700e00640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.812+0000 7f2700e00640 1 -- 172.21.15.62:0/2901130738 >> v1:172.21.15.62:6789/0 conn(0x7f26fc0ae370 legacy=0x7f26fc0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.813+0000 7f2700e00640 1 -- 172.21.15.62:0/2901130738 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0b3970 con 0x7f26fc0719b0 2024-03-28T18:02:52.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.813+0000 7f26fb400640 1 -- 172.21.15.62:0/2901130738 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307159241 0 0) 0x7f26fc0b30d0 con 0x7f26fc0ae370 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.814+0000 7f2700e00640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0b4170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb41f7af82f76d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.814+0000 7f26fb400640 1 -- 172.21.15.62:0/2901130738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec039070 con 0x7f26fc0719b0 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.814+0000 7f26fb400640 1 -- 172.21.15.62:0/2901130738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec02cc60 con 0x7f26fc0719b0 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.814+0000 7f26fb400640 1 -- 172.21.15.62:0/2901130738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec034660 con 0x7f26fc0719b0 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 -- 172.21.15.62:0/2901130738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 msgr2=0x7f26fc0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0b4170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec02beb0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 -- 172.21.15.62:0/2901130738 shutdown_connections 2024-03-28T18:02:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 --2- 172.21.15.62:0/2901130738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0b4170 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 -- 172.21.15.62:0/2901130738 >> 172.21.15.62:0/2901130738 conn(0x7f26fc066fb0 msgr2=0x7f26fc0673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 -- 172.21.15.62:0/2901130738 shutdown_connections 2024-03-28T18:02:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.815+0000 7f2703800640 1 -- 172.21.15.62:0/2901130738 wait complete. 2024-03-28T18:02:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.817+0000 7f2703800640 1 Processor -- start 2024-03-28T18:02:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.817+0000 7f2703800640 1 -- start start 2024-03-28T18:02:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 --2- >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 0x7f26fc14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b46b0 con 0x7f26fc0719b0 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc072760 con 0x7f26fc0ae370 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2703800640 1 -- --> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc06ccb0 con 0x7f26fc071d90 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2700e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2700e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33592/0 (socket says 172.21.15.62:33592) 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f2700e00640 1 -- 172.21.15.62:0/1465281134 learned_addr learned my addr 172.21.15.62:0/1465281134 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.818+0000 7f26fbe00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2701800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 0x7f26fc14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 -- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 msgr2=0x7f26fc14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc14ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 -- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 msgr2=0x7f26fc14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 0x7f26fc14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 -- 172.21.15.62:0/1465281134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0c9760 con 0x7f26fc0719b0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2701800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 0x7f26fc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f26fbe00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2700e00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec039050 con 0x7f26fc0719b0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec045070 con 0x7f26fc0719b0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.819+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc0c9a40 con 0x7f26fc0719b0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.820+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec03e850 con 0x7f26fc0719b0 2024-03-28T18:02:52.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.820+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc0ca000 con 0x7f26fc0719b0 2024-03-28T18:02:52.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.821+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc0680c0 con 0x7f26fc0719b0 2024-03-28T18:02:52.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.823+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f26ec03e9f0 con 0x7f26fc0719b0 2024-03-28T18:02:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.824+0000 7f26f8c00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 0x7f26d806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.824+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26ec0ad930 con 0x7f26fc0719b0 2024-03-28T18:02:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.825+0000 7f26fbe00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 0x7f26d806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.826+0000 7f26fbe00640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 0x7f26d806e090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26fc15c520 tx=0x7f26e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:52.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:52.833+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (secure 0 0 0) 0x7f26ec07c530 con 0x7f26fc0719b0 2024-03-28T18:02:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.190+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) v1 -- 0x7f26fc0b0b90 con 0x7f26fc0719b0 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.191+0000 7f26f8c00640 1 -- 172.21.15.62:0/1465281134 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v9) v1 ==== 82+0+151 (secure 0 0 0) 0x7f26ec07be30 con 0x7f26fc0719b0 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stdout:[client.admin] 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCarwVmIZL/KBAAmaVMTNm3m+cgI0ksd6ObvQ== 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow *" 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stdout: caps mgr = "allow *" 2024-03-28T18:02:53.191 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow *" 2024-03-28T18:02:53.192 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow *" 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 msgr2=0x7f26d806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 0x7f26d806e090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26fc15c520 tx=0x7f26e4009040 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 msgr2=0x7f26fc0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec004a40 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 shutdown_connections 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.193+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:6832/1316232289,v1:172.21.15.81:6833/1316232289] conn(0x7f26d806bbd0 0x7f26d806e090 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26fc0ae370 0x7f26fc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f26fc071d90 0x7f26fc14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 --2- 172.21.15.62:0/1465281134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0719b0 0x7f26fc0c9030 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 >> 172.21.15.62:0/1465281134 conn(0x7f26fc066fb0 msgr2=0x7f26fc0ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:53.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 shutdown_connections 2024-03-28T18:02:53.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-28T18:02:53.194+0000 7f2703800640 1 -- 172.21.15.62:0/1465281134 wait complete. 2024-03-28T18:02:53.213 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-03-28T18:02:53.252 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.KZ0wtCVxxP 2024-03-28T18:02:53.252 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T18:02:53.252 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/tmp/tmp.KZ0wtCVxxP 2024-03-28T18:02:53.338 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T18:02:53.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/tmp/tmp.7I1MPLSTiQxfstests-dev/local.config 2024-03-28T18:02:53.475 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.sy6xOdcDqftest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.NdNTLS4TFuscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.62:6789 -k /tmp/tmp.KZ0wtCVxxP --client_mountpoint /test" 2024-03-28T18:02:53.476 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-28T18:02:53.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/tmp/tmp.7I1MPLSTiQxfstests-dev/ceph.exclude 2024-03-28T18:02:53.591 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo make) 2024-03-28T18:02:53.712 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize -cfi 2024-03-28T18:02:54.773 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-28T18:02:54.774 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file './config.guess' 2024-03-28T18:02:54.788 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file './config.sub' 2024-03-28T18:02:54.802 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file './install-sh' 2024-03-28T18:02:54.815 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file './ltmain.sh' 2024-03-28T18:02:54.899 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-28T18:02:54.900 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-28T18:02:54.918 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-28T18:02:54.936 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-28T18:02:54.954 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-28T18:02:54.970 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-28T18:02:54.994 INFO:teuthology.orchestra.run.smithi062.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-28T18:02:54.998 INFO:teuthology.orchestra.run.smithi062.stdout:cp include/install-sh . 2024-03-28T18:02:55.000 INFO:teuthology.orchestra.run.smithi062.stdout:aclocal -I m4 2024-03-28T18:02:56.476 INFO:teuthology.orchestra.run.smithi062.stdout:autoheader 2024-03-28T18:02:56.913 INFO:teuthology.orchestra.run.smithi062.stdout:autoconf 2024-03-28T18:02:57.173 INFO:teuthology.orchestra.run.smithi062.stdout:./configure \ 2024-03-28T18:02:57.173 INFO:teuthology.orchestra.run.smithi062.stdout: --libexecdir=/usr/lib \ 2024-03-28T18:02:57.173 INFO:teuthology.orchestra.run.smithi062.stdout: --exec_prefix=/var/lib 2024-03-28T18:02:57.627 INFO:teuthology.orchestra.run.smithi062.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-28T18:02:57.627 INFO:teuthology.orchestra.run.smithi062.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-28T18:02:57.631 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to print strings... printf 2024-03-28T18:02:57.631 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gcc... gcc 2024-03-28T18:02:57.763 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether the C compiler works... yes 2024-03-28T18:02:57.763 INFO:teuthology.orchestra.run.smithi062.stdout:checking for C compiler default output file name... a.out 2024-03-28T18:02:57.836 INFO:teuthology.orchestra.run.smithi062.stdout:checking for suffix of executables... 2024-03-28T18:02:57.967 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether we are cross compiling... no 2024-03-28T18:02:58.019 INFO:teuthology.orchestra.run.smithi062.stdout:checking for suffix of object files... o 2024-03-28T18:02:58.063 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether we are using the GNU C compiler... yes 2024-03-28T18:02:58.105 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether gcc accepts -g... yes 2024-03-28T18:02:58.163 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-28T18:02:58.172 INFO:teuthology.orchestra.run.smithi062.stdout:checking for a sed that does not truncate output... /bin/sed 2024-03-28T18:02:58.179 INFO:teuthology.orchestra.run.smithi062.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-03-28T18:02:58.183 INFO:teuthology.orchestra.run.smithi062.stdout:checking for egrep... /bin/grep -E 2024-03-28T18:02:58.188 INFO:teuthology.orchestra.run.smithi062.stdout:checking for fgrep... /bin/grep -F 2024-03-28T18:02:58.198 INFO:teuthology.orchestra.run.smithi062.stdout:checking for ld used by gcc... /bin/ld 2024-03-28T18:02:58.203 INFO:teuthology.orchestra.run.smithi062.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-03-28T18:02:58.220 INFO:teuthology.orchestra.run.smithi062.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-03-28T18:02:59.290 INFO:teuthology.orchestra.run.smithi062.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-03-28T18:02:59.291 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether ln -s works... yes 2024-03-28T18:02:59.312 INFO:teuthology.orchestra.run.smithi062.stdout:checking the maximum length of command line arguments... 1572864 2024-03-28T18:02:59.323 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-28T18:02:59.324 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-28T18:02:59.324 INFO:teuthology.orchestra.run.smithi062.stdout:checking for /bin/ld option to reload object files... -r 2024-03-28T18:02:59.326 INFO:teuthology.orchestra.run.smithi062.stdout:checking for objdump... objdump 2024-03-28T18:02:59.327 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to recognize dependent libraries... pass_all 2024-03-28T18:02:59.329 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dlltool... no 2024-03-28T18:02:59.329 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-28T18:02:59.331 INFO:teuthology.orchestra.run.smithi062.stdout:checking for ar... ar 2024-03-28T18:02:59.452 INFO:teuthology.orchestra.run.smithi062.stdout:checking for archiver @FILE support... @ 2024-03-28T18:02:59.453 INFO:teuthology.orchestra.run.smithi062.stdout:checking for strip... strip 2024-03-28T18:02:59.455 INFO:teuthology.orchestra.run.smithi062.stdout:checking for ranlib... ranlib 2024-03-28T18:02:59.456 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gawk... gawk 2024-03-28T18:02:59.656 INFO:teuthology.orchestra.run.smithi062.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-03-28T18:02:59.664 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sysroot... no 2024-03-28T18:02:59.679 INFO:teuthology.orchestra.run.smithi062.stdout:checking for a working dd... /bin/dd 2024-03-28T18:02:59.693 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-03-28T18:02:59.741 INFO:teuthology.orchestra.run.smithi062.stdout:checking for mt... no 2024-03-28T18:02:59.753 INFO:teuthology.orchestra.run.smithi062.stdout:checking if : is a manifest tool... no 2024-03-28T18:02:59.867 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to run the C preprocessor... gcc -E 2024-03-28T18:03:00.229 INFO:teuthology.orchestra.run.smithi062.stdout:checking for ANSI C header files... yes 2024-03-28T18:03:00.280 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/types.h... yes 2024-03-28T18:03:00.348 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/stat.h... yes 2024-03-28T18:03:00.411 INFO:teuthology.orchestra.run.smithi062.stdout:checking for stdlib.h... yes 2024-03-28T18:03:00.475 INFO:teuthology.orchestra.run.smithi062.stdout:checking for string.h... yes 2024-03-28T18:03:00.543 INFO:teuthology.orchestra.run.smithi062.stdout:checking for memory.h... yes 2024-03-28T18:03:00.609 INFO:teuthology.orchestra.run.smithi062.stdout:checking for strings.h... yes 2024-03-28T18:03:00.688 INFO:teuthology.orchestra.run.smithi062.stdout:checking for inttypes.h... yes 2024-03-28T18:03:00.759 INFO:teuthology.orchestra.run.smithi062.stdout:checking for stdint.h... yes 2024-03-28T18:03:00.847 INFO:teuthology.orchestra.run.smithi062.stdout:checking for unistd.h... yes 2024-03-28T18:03:00.922 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dlfcn.h... yes 2024-03-28T18:03:00.937 INFO:teuthology.orchestra.run.smithi062.stdout:checking for objdir... .libs 2024-03-28T18:03:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-28T18:03:01.111 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-28T18:03:01.173 INFO:teuthology.orchestra.run.smithi062.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-28T18:03:01.224 INFO:teuthology.orchestra.run.smithi062.stdout:checking if gcc static flag -static works... no 2024-03-28T18:03:01.292 INFO:teuthology.orchestra.run.smithi062.stdout:checking if gcc supports -c -o file.o... yes 2024-03-28T18:03:01.293 INFO:teuthology.orchestra.run.smithi062.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-28T18:03:01.315 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-28T18:03:01.389 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether -lc should be explicitly linked in... no 2024-03-28T18:03:01.537 INFO:teuthology.orchestra.run.smithi062.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-28T18:03:01.538 INFO:teuthology.orchestra.run.smithi062.stdout:checking how to hardcode library paths into programs... immediate 2024-03-28T18:03:01.544 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether stripping libraries is possible... yes 2024-03-28T18:03:01.544 INFO:teuthology.orchestra.run.smithi062.stdout:checking if libtool supports shared libraries... yes 2024-03-28T18:03:01.544 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether to build shared libraries... yes 2024-03-28T18:03:01.544 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether to build static libraries... yes 2024-03-28T18:03:01.570 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gcc... (cached) gcc 2024-03-28T18:03:01.633 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-28T18:03:01.634 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-28T18:03:01.635 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-28T18:03:01.636 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gmake... /usr/bin/gmake 2024-03-28T18:03:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-03-28T18:03:01.637 INFO:teuthology.orchestra.run.smithi062.stdout:checking for tar... /bin/tar 2024-03-28T18:03:01.637 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gzip... /bin/gzip 2024-03-28T18:03:01.665 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether gcc -MM is supported... yes 2024-03-28T18:03:01.666 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sort... /bin/sort 2024-03-28T18:03:01.666 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether ln -s works... yes 2024-03-28T18:03:01.667 INFO:teuthology.orchestra.run.smithi062.stdout:checking for rpm... /bin/rpm 2024-03-28T18:03:01.678 INFO:teuthology.orchestra.run.smithi062.stdout:checking for rpmbuild... no 2024-03-28T18:03:01.743 INFO:teuthology.orchestra.run.smithi062.stdout:checking assert.h usability... yes 2024-03-28T18:03:01.768 INFO:teuthology.orchestra.run.smithi062.stdout:checking assert.h presence... yes 2024-03-28T18:03:01.768 INFO:teuthology.orchestra.run.smithi062.stdout:checking for assert.h... yes 2024-03-28T18:03:01.839 INFO:teuthology.orchestra.run.smithi062.stdout:checking bstring.h usability... no 2024-03-28T18:03:01.875 INFO:teuthology.orchestra.run.smithi062.stdout:checking bstring.h presence... no 2024-03-28T18:03:01.875 INFO:teuthology.orchestra.run.smithi062.stdout:checking for bstring.h... no 2024-03-28T18:03:01.943 INFO:teuthology.orchestra.run.smithi062.stdout:checking libgen.h usability... yes 2024-03-28T18:03:01.971 INFO:teuthology.orchestra.run.smithi062.stdout:checking libgen.h presence... yes 2024-03-28T18:03:01.971 INFO:teuthology.orchestra.run.smithi062.stdout:checking for libgen.h... yes 2024-03-28T18:03:02.061 INFO:teuthology.orchestra.run.smithi062.stdout:checking dirent.h usability... yes 2024-03-28T18:03:02.092 INFO:teuthology.orchestra.run.smithi062.stdout:checking dirent.h presence... yes 2024-03-28T18:03:02.092 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dirent.h... yes 2024-03-28T18:03:02.189 INFO:teuthology.orchestra.run.smithi062.stdout:checking errno.h usability... yes 2024-03-28T18:03:02.215 INFO:teuthology.orchestra.run.smithi062.stdout:checking errno.h presence... yes 2024-03-28T18:03:02.215 INFO:teuthology.orchestra.run.smithi062.stdout:checking for errno.h... yes 2024-03-28T18:03:02.292 INFO:teuthology.orchestra.run.smithi062.stdout:checking malloc.h usability... yes 2024-03-28T18:03:02.325 INFO:teuthology.orchestra.run.smithi062.stdout:checking malloc.h presence... yes 2024-03-28T18:03:02.326 INFO:teuthology.orchestra.run.smithi062.stdout:checking for malloc.h... yes 2024-03-28T18:03:02.408 INFO:teuthology.orchestra.run.smithi062.stdout:checking uuid.h usability... no 2024-03-28T18:03:02.451 INFO:teuthology.orchestra.run.smithi062.stdout:checking uuid.h presence... no 2024-03-28T18:03:02.452 INFO:teuthology.orchestra.run.smithi062.stdout:checking for uuid.h... no 2024-03-28T18:03:02.545 INFO:teuthology.orchestra.run.smithi062.stdout:checking uuid/uuid.h usability... yes 2024-03-28T18:03:02.578 INFO:teuthology.orchestra.run.smithi062.stdout:checking uuid/uuid.h presence... yes 2024-03-28T18:03:02.578 INFO:teuthology.orchestra.run.smithi062.stdout:checking for uuid/uuid.h... yes 2024-03-28T18:03:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/uuid.h usability... no 2024-03-28T18:03:02.687 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/uuid.h presence... no 2024-03-28T18:03:02.687 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/uuid.h... no 2024-03-28T18:03:02.753 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/file.h usability... yes 2024-03-28T18:03:02.779 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/file.h presence... yes 2024-03-28T18:03:02.779 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/file.h... yes 2024-03-28T18:03:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fcntl.h usability... yes 2024-03-28T18:03:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fcntl.h presence... yes 2024-03-28T18:03:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/fcntl.h... yes 2024-03-28T18:03:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/syssgi.h usability... no 2024-03-28T18:03:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/syssgi.h presence... no 2024-03-28T18:03:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/syssgi.h... no 2024-03-28T18:03:03.079 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/param.h usability... yes 2024-03-28T18:03:03.111 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/param.h presence... yes 2024-03-28T18:03:03.111 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/param.h... yes 2024-03-28T18:03:03.127 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/stat.h... (cached) yes 2024-03-28T18:03:03.200 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/statvfs.h usability... yes 2024-03-28T18:03:03.229 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/statvfs.h presence... yes 2024-03-28T18:03:03.230 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/statvfs.h... yes 2024-03-28T18:03:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/time.h usability... yes 2024-03-28T18:03:03.331 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/time.h presence... yes 2024-03-28T18:03:03.331 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/time.h... yes 2024-03-28T18:03:03.417 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/ioctl.h usability... yes 2024-03-28T18:03:03.445 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/ioctl.h presence... yes 2024-03-28T18:03:03.445 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/ioctl.h... yes 2024-03-28T18:03:03.527 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/wait.h usability... yes 2024-03-28T18:03:03.560 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/wait.h presence... yes 2024-03-28T18:03:03.561 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/wait.h... yes 2024-03-28T18:03:03.580 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/types.h... (cached) yes 2024-03-28T18:03:03.596 INFO:teuthology.orchestra.run.smithi062.stdout:checking for strings.h... (cached) yes 2024-03-28T18:03:03.660 INFO:teuthology.orchestra.run.smithi062.stdout:checking err.h usability... yes 2024-03-28T18:03:03.686 INFO:teuthology.orchestra.run.smithi062.stdout:checking err.h presence... yes 2024-03-28T18:03:03.686 INFO:teuthology.orchestra.run.smithi062.stdout:checking for err.h... yes 2024-03-28T18:03:03.757 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/falloc.h usability... yes 2024-03-28T18:03:03.782 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/falloc.h presence... yes 2024-03-28T18:03:03.782 INFO:teuthology.orchestra.run.smithi062.stdout:checking for linux/falloc.h... yes 2024-03-28T18:03:03.850 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-28T18:03:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-28T18:03:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-28T18:03:03.946 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-28T18:03:03.979 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-28T18:03:03.979 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-28T18:03:04.047 INFO:teuthology.orchestra.run.smithi062.stdout:checking xfs/platform_defs.h usability... no 2024-03-28T18:03:04.084 INFO:teuthology.orchestra.run.smithi062.stdout:checking xfs/platform_defs.h presence... no 2024-03-28T18:03:04.084 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/platform_defs.h... no 2024-03-28T18:03:04.152 INFO:teuthology.orchestra.run.smithi062.stdout:checking btrfs/ioctl.h usability... no 2024-03-28T18:03:04.192 INFO:teuthology.orchestra.run.smithi062.stdout:checking btrfs/ioctl.h presence... no 2024-03-28T18:03:04.192 INFO:teuthology.orchestra.run.smithi062.stdout:checking for btrfs/ioctl.h... no 2024-03-28T18:03:04.258 INFO:teuthology.orchestra.run.smithi062.stdout:checking cifs/ioctl.h usability... no 2024-03-28T18:03:04.292 INFO:teuthology.orchestra.run.smithi062.stdout:checking cifs/ioctl.h presence... no 2024-03-28T18:03:04.292 INFO:teuthology.orchestra.run.smithi062.stdout:checking for cifs/ioctl.h... no 2024-03-28T18:03:04.357 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/mman.h usability... yes 2024-03-28T18:03:04.386 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/mman.h presence... yes 2024-03-28T18:03:04.386 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/mman.h... yes 2024-03-28T18:03:04.435 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/xfs_log_format.h... no 2024-03-28T18:03:04.531 INFO:teuthology.orchestra.run.smithi062.stdout:checking for uuid_compare... no 2024-03-28T18:03:04.613 INFO:teuthology.orchestra.run.smithi062.stdout:checking for uuid_compare in -luuid... yes 2024-03-28T18:03:04.750 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/xfs.h... yes 2024-03-28T18:03:04.795 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/libxfs.h... no 2024-03-28T18:03:04.849 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-28T18:03:04.922 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/xqm.h... yes 2024-03-28T18:03:05.036 INFO:teuthology.orchestra.run.smithi062.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-28T18:03:05.099 INFO:teuthology.orchestra.run.smithi062.stdout:checking xfs/handle.h usability... yes 2024-03-28T18:03:05.126 INFO:teuthology.orchestra.run.smithi062.stdout:checking xfs/handle.h presence... yes 2024-03-28T18:03:05.126 INFO:teuthology.orchestra.run.smithi062.stdout:checking for xfs/handle.h... yes 2024-03-28T18:03:05.212 INFO:teuthology.orchestra.run.smithi062.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-28T18:03:05.272 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/acl.h usability... yes 2024-03-28T18:03:05.303 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/acl.h presence... yes 2024-03-28T18:03:05.303 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/acl.h... yes 2024-03-28T18:03:05.365 INFO:teuthology.orchestra.run.smithi062.stdout:checking acl/libacl.h usability... yes 2024-03-28T18:03:05.395 INFO:teuthology.orchestra.run.smithi062.stdout:checking acl/libacl.h presence... yes 2024-03-28T18:03:05.395 INFO:teuthology.orchestra.run.smithi062.stdout:checking for acl/libacl.h... yes 2024-03-28T18:03:05.472 INFO:teuthology.orchestra.run.smithi062.stdout:checking for acl_init in -lacl... yes 2024-03-28T18:03:05.538 INFO:teuthology.orchestra.run.smithi062.stdout:checking gdbm-ndbm.h usability... no 2024-03-28T18:03:05.581 INFO:teuthology.orchestra.run.smithi062.stdout:checking gdbm-ndbm.h presence... no 2024-03-28T18:03:05.581 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gdbm-ndbm.h... no 2024-03-28T18:03:05.648 INFO:teuthology.orchestra.run.smithi062.stdout:checking gdbm/ndbm.h usability... yes 2024-03-28T18:03:05.678 INFO:teuthology.orchestra.run.smithi062.stdout:checking gdbm/ndbm.h presence... yes 2024-03-28T18:03:05.678 INFO:teuthology.orchestra.run.smithi062.stdout:checking for gdbm/ndbm.h... yes 2024-03-28T18:03:05.772 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_open in -lgdbm... no 2024-03-28T18:03:05.865 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_fetch in -lgdbm... no 2024-03-28T18:03:05.960 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_store in -lgdbm... no 2024-03-28T18:03:06.052 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_close in -lgdbm... no 2024-03-28T18:03:06.158 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-03-28T18:03:06.239 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-03-28T18:03:06.316 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-03-28T18:03:06.402 INFO:teuthology.orchestra.run.smithi062.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-03-28T18:03:06.465 INFO:teuthology.orchestra.run.smithi062.stdout:checking libaio.h usability... yes 2024-03-28T18:03:06.499 INFO:teuthology.orchestra.run.smithi062.stdout:checking libaio.h presence... yes 2024-03-28T18:03:06.499 INFO:teuthology.orchestra.run.smithi062.stdout:checking for libaio.h... yes 2024-03-28T18:03:06.565 INFO:teuthology.orchestra.run.smithi062.stdout:checking liburing.h usability... no 2024-03-28T18:03:06.604 INFO:teuthology.orchestra.run.smithi062.stdout:checking liburing.h presence... no 2024-03-28T18:03:06.604 INFO:teuthology.orchestra.run.smithi062.stdout:checking for liburing.h... no 2024-03-28T18:03:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/capability.h usability... yes 2024-03-28T18:03:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:checking sys/capability.h presence... yes 2024-03-28T18:03:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:checking for sys/capability.h... yes 2024-03-28T18:03:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/fiemap.h usability... yes 2024-03-28T18:03:06.798 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/fiemap.h presence... yes 2024-03-28T18:03:06.798 INFO:teuthology.orchestra.run.smithi062.stdout:checking for linux/fiemap.h... yes 2024-03-28T18:03:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:checking for fallocate... yes 2024-03-28T18:03:06.966 INFO:teuthology.orchestra.run.smithi062.stdout:checking for open_by_handle_at... yes 2024-03-28T18:03:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/fs.h usability... yes 2024-03-28T18:03:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/fs.h presence... yes 2024-03-28T18:03:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:checking for linux/fs.h... yes 2024-03-28T18:03:07.121 INFO:teuthology.orchestra.run.smithi062.stdout:checking btrfsutil.h usability... no 2024-03-28T18:03:07.163 INFO:teuthology.orchestra.run.smithi062.stdout:checking btrfsutil.h presence... no 2024-03-28T18:03:07.163 INFO:teuthology.orchestra.run.smithi062.stdout:checking for btrfsutil.h... no 2024-03-28T18:03:07.262 INFO:teuthology.orchestra.run.smithi062.stdout:checking for copy_file_range... yes 2024-03-28T18:03:07.348 INFO:teuthology.orchestra.run.smithi062.stdout:checking for SEEK_DATA... yes 2024-03-28T18:03:07.458 INFO:teuthology.orchestra.run.smithi062.stdout:checking for BMV_OF_SHARED... yes 2024-03-28T18:03:07.542 INFO:teuthology.orchestra.run.smithi062.stdout:checking for nftw... yes 2024-03-28T18:03:07.643 INFO:teuthology.orchestra.run.smithi062.stdout:checking for RLIMIT_NOFILE... yes 2024-03-28T18:03:07.727 INFO:teuthology.orchestra.run.smithi062.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-03-28T18:03:07.815 INFO:teuthology.orchestra.run.smithi062.stdout:checking for FICLONE... yes 2024-03-28T18:03:07.902 INFO:teuthology.orchestra.run.smithi062.stdout:checking for renameat2... yes 2024-03-28T18:03:07.990 INFO:teuthology.orchestra.run.smithi062.stdout:checking for reallocarray... yes 2024-03-28T18:03:08.093 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct mount_attr... yes 2024-03-28T18:03:08.211 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_qgroup_limit... yes 2024-03-28T18:03:08.304 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-03-28T18:03:08.401 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-03-28T18:03:08.498 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-03-28T18:03:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-03-28T18:03:08.690 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-03-28T18:03:08.787 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-03-28T18:03:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/btrfs.h usability... yes 2024-03-28T18:03:08.888 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/btrfs.h presence... yes 2024-03-28T18:03:08.888 INFO:teuthology.orchestra.run.smithi062.stdout:checking for linux/btrfs.h... yes 2024-03-28T18:03:08.974 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/btrfs_tree.h usability... yes 2024-03-28T18:03:09.003 INFO:teuthology.orchestra.run.smithi062.stdout:checking linux/btrfs_tree.h presence... yes 2024-03-28T18:03:09.003 INFO:teuthology.orchestra.run.smithi062.stdout:checking for linux/btrfs_tree.h... yes 2024-03-28T18:03:09.067 INFO:teuthology.orchestra.run.smithi062.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-03-28T18:03:09.145 INFO:teuthology.orchestra.run.smithi062.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-03-28T18:03:09.181 INFO:teuthology.orchestra.run.smithi062.stdout:configure: creating ./config.status 2024-03-28T18:03:10.264 INFO:teuthology.orchestra.run.smithi062.stdout:config.status: creating include/builddefs 2024-03-28T18:03:10.304 INFO:teuthology.orchestra.run.smithi062.stdout:config.status: creating include/config.h 2024-03-28T18:03:10.333 INFO:teuthology.orchestra.run.smithi062.stdout:config.status: executing libtool commands 2024-03-28T18:03:10.450 INFO:teuthology.orchestra.run.smithi062.stdout:Building include 2024-03-28T18:03:10.460 INFO:teuthology.orchestra.run.smithi062.stdout:Building lib 2024-03-28T18:03:10.469 INFO:teuthology.orchestra.run.smithi062.stdout: [LTDEP] 2024-03-28T18:03:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dataascii.lo 2024-03-28T18:03:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] databin.lo 2024-03-28T18:03:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] datapid.lo 2024-03-28T18:03:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] file_lock.lo 2024-03-28T18:03:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] forker.lo 2024-03-28T18:03:12.250 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] pattern.lo 2024-03-28T18:03:12.496 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] open_flags.lo 2024-03-28T18:03:12.795 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] random_range.lo 2024-03-28T18:03:13.179 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] string_to_tokens.lo 2024-03-28T18:03:13.421 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] str_to_bytes.lo 2024-03-28T18:03:13.688 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] tlibio.lo 2024-03-28T18:03:14.102 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] write_log.lo 2024-03-28T18:03:14.535 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] random.lo 2024-03-28T18:03:14.780 INFO:teuthology.orchestra.run.smithi062.stdout: [LD] 2024-03-28T18:03:14.997 INFO:teuthology.orchestra.run.smithi062.stdout:Building ltp 2024-03-28T18:03:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:15.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:03:15.173 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:03:15.358 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] doio 2024-03-28T18:03:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fsstress 2024-03-28T18:03:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fsx 2024-03-28T18:03:18.920 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] growfiles 2024-03-28T18:03:19.607 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] iogen 2024-03-28T18:03:20.233 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-stress 2024-03-28T18:03:20.876 INFO:teuthology.orchestra.run.smithi062.stdout:Building src 2024-03-28T18:03:20.890 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: [DEP] 2024-03-28T18:03:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dirstress 2024-03-28T18:03:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fill 2024-03-28T18:03:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fill2 2024-03-28T18:03:26.924 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] getpagesize 2024-03-28T18:03:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] holes 2024-03-28T18:03:27.549 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] lstat64 2024-03-28T18:03:27.879 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] nametest 2024-03-28T18:03:28.278 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] permname 2024-03-28T18:03:28.638 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] randholes 2024-03-28T18:03:29.083 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] runas 2024-03-28T18:03:29.475 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] truncfile 2024-03-28T18:03:29.797 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] usemem 2024-03-28T18:03:30.097 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] mmapcat 2024-03-28T18:03:30.435 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] append_reader 2024-03-28T18:03:30.769 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] append_writer 2024-03-28T18:03:31.101 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dirperf 2024-03-28T18:03:31.546 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] metaperf 2024-03-28T18:03:31.992 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] devzero 2024-03-28T18:03:32.335 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] feature 2024-03-28T18:03:32.717 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] alloc 2024-03-28T18:03:33.090 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fault 2024-03-28T18:03:33.404 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fstest 2024-03-28T18:03:33.814 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_access_root 2024-03-28T18:03:34.108 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] godown 2024-03-28T18:03:34.455 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] resvtest 2024-03-28T18:03:34.802 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] writemod 2024-03-28T18:03:35.134 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] writev_on_pagefault 2024-03-28T18:03:35.475 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] makeextents 2024-03-28T18:03:35.859 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] itrash 2024-03-28T18:03:36.247 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] rename 2024-03-28T18:03:36.522 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] multi_open_unlink 2024-03-28T18:03:36.897 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] unwritten_sync 2024-03-28T18:03:37.285 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] genhashnames 2024-03-28T18:03:37.668 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_holes 2024-03-28T18:03:37.945 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_writev 2024-03-28T18:03:38.252 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_truncate_cmtime 2024-03-28T18:03:38.564 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dirhash_collide 2024-03-28T18:03:38.891 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_rename_overwrite 2024-03-28T18:03:39.264 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] holetest 2024-03-28T18:03:39.641 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_truncate_self 2024-03-28T18:03:39.893 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] af_unix 2024-03-28T18:03:40.198 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_stale_pmd 2024-03-28T18:03:40.503 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_cow_race 2024-03-28T18:03:40.815 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_fallocate 2024-03-28T18:03:41.114 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fsync-err 2024-03-28T18:03:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_write_ro 2024-03-28T18:03:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_ext4_dax_journal_corruption 2024-03-28T18:03:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_ext4_dax_inline_corruption 2024-03-28T18:03:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_ofd_locks 2024-03-28T18:03:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_collision 2024-03-28T18:03:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] mmap-write-concurrent 2024-03-28T18:03:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_get_file_time 2024-03-28T18:03:43.799 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_create_short_dirs 2024-03-28T18:03:44.133 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_create_long_dirs 2024-03-28T18:03:44.461 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_enospc 2024-03-28T18:03:44.844 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_writev_overlap 2024-03-28T18:03:45.165 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] checkpoint_journal 2024-03-28T18:03:45.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:03:45.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:03:45.477 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] mmap-rw-fault 2024-03-28T18:03:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] allocstale 2024-03-28T18:03:46.143 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_cow_memory_failure 2024-03-28T18:03:46.452 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fake-dump-rootino 2024-03-28T18:03:46.735 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dio-buf-fault 2024-03-28T18:03:47.039 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] rewinddir-test 2024-03-28T18:03:47.332 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] readdir-while-renames 2024-03-28T18:03:47.620 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] xfsctl 2024-03-28T18:03:47.945 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] bstat 2024-03-28T18:03:48.330 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mtab 2024-03-28T18:03:48.716 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] getdevicesize 2024-03-28T18:03:49.047 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] preallo_rw_pattern_reader 2024-03-28T18:03:49.352 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] preallo_rw_pattern_writer 2024-03-28T18:03:49.656 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] ftrunc 2024-03-28T18:03:49.956 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] trunc 2024-03-28T18:03:50.270 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fs_perms 2024-03-28T18:03:50.589 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] testx 2024-03-28T18:03:50.865 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] looptest 2024-03-28T18:03:51.252 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] locktest 2024-03-28T18:03:51.719 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] unwritten_mmap 2024-03-28T18:03:52.021 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] bulkstat_unlink_test 2024-03-28T18:03:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] deduperace 2024-03-28T18:03:52.754 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] bulkstat_unlink_test_modified 2024-03-28T18:03:53.102 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_dir_offset 2024-03-28T18:03:53.396 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_futimens 2024-03-28T18:03:53.697 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_immutable 2024-03-28T18:03:54.632 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] stale_handle 2024-03-28T18:03:54.953 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] pwrite_mmap_blocked 2024-03-28T18:03:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_dir_offset2 2024-03-28T18:03:55.604 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] seek_sanity_test 2024-03-28T18:03:56.305 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] seek_copy_test 2024-03-28T18:03:56.615 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_readdir_1 2024-03-28T18:03:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_readdir_2 2024-03-28T18:03:57.153 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fsync-tester 2024-03-28T18:03:57.534 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] nsexec 2024-03-28T18:03:57.806 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] cloner 2024-03-28T18:03:58.099 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] renameat2 2024-03-28T18:03:58.405 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_getcwd 2024-03-28T18:03:58.739 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] e4compact 2024-03-28T18:03:59.090 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] test-nextquota 2024-03-28T18:03:59.417 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] punch-alternating 2024-03-28T18:03:59.733 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] attr-list-by-handle-cursor-test 2024-03-28T18:04:00.056 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] listxattr 2024-03-28T18:04:00.331 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dio-interleaved 2024-03-28T18:04:00.649 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_dir_type 2024-03-28T18:04:00.939 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dio-invalidate-cache 2024-03-28T18:04:01.320 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] stat_test 2024-03-28T18:04:01.802 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_encrypted_d_revalidate 2024-03-28T18:04:02.173 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] attr_replace_test 2024-03-28T18:04:02.474 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] swapon 2024-03-28T18:04:02.848 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] mkswap 2024-03-28T18:04:03.253 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_attr_corruption 2024-03-28T18:04:03.545 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_open_tmpfiles 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fscrypt-crypt-util 2024-03-28T18:04:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] bulkstat_null_ocount 2024-03-28T18:04:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] splice-test 2024-03-28T18:04:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] chprojid_fail 2024-03-28T18:04:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] detached_mounts_propagation 2024-03-28T18:04:06.220 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] ext4_resize 2024-03-28T18:04:06.505 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_readdir_3 2024-03-28T18:04:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] splice2pipe 2024-03-28T18:04:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] uuid_ioctl 2024-03-28T18:04:07.583 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_snapshot_deleted_subvolume 2024-03-28T18:04:07.893 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fiemap-tester 2024-03-28T18:04:08.328 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_stripealign 2024-03-28T18:04:08.640 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] open_by_handle 2024-03-28T18:04:09.040 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] dbtest 2024-03-28T18:04:09.471 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_mmap_dio 2024-03-28T18:04:09.789 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] xfsfind 2024-03-28T18:04:10.158 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] t_reflink_read_race 2024-03-28T18:04:10.568 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] fssum 2024-03-28T18:04:11.190 INFO:teuthology.orchestra.run.smithi062.stdout:Building log-writes 2024-03-28T18:04:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] replay-log 2024-03-28T18:04:11.734 INFO:teuthology.orchestra.run.smithi062.stdout:Building perf 2024-03-28T18:04:11.745 INFO:teuthology.orchestra.run.smithi062.stdout:Building aio-dio-regress 2024-03-28T18:04:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aiocp 2024-03-28T18:04:12.161 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-append-write-fallocate-race 2024-03-28T18:04:12.566 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-append-write-read-race 2024-03-28T18:04:12.872 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-cow-race 2024-03-28T18:04:13.163 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-cycle-write 2024-03-28T18:04:13.489 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-eof-race 2024-03-28T18:04:13.819 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-extend-stat 2024-03-28T18:04:14.130 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-fcntl-race 2024-03-28T18:04:14.445 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-hole-filling-race 2024-03-28T18:04:14.742 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-invalidate-failure 2024-03-28T18:04:15.067 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-invalidate-readahead 2024-03-28T18:04:15.409 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aiodio_sparse2 2024-03-28T18:04:15.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:04:15.506 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:04:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-subblock-eof-read 2024-03-28T18:04:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-dio-write-verify 2024-03-28T18:04:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-03-28T18:04:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-03-28T18:04:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] aio-last-ref-held-by-io 2024-03-28T18:04:17.311 INFO:teuthology.orchestra.run.smithi062.stdout:Building vfs 2024-03-28T18:04:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] vfstest 2024-03-28T18:04:21.466 INFO:teuthology.orchestra.run.smithi062.stdout: [CC] mount-idmapped 2024-03-28T18:04:22.034 INFO:teuthology.orchestra.run.smithi062.stdout:Building m4 2024-03-28T18:04:22.045 INFO:teuthology.orchestra.run.smithi062.stdout:Building common 2024-03-28T18:04:22.055 INFO:teuthology.orchestra.run.smithi062.stdout:Building tests 2024-03-28T18:04:22.069 INFO:teuthology.orchestra.run.smithi062.stdout:Building btrfs 2024-03-28T18:04:22.084 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/btrfs/group.list 2024-03-28T18:04:22.316 INFO:teuthology.orchestra.run.smithi062.stdout:Building ceph 2024-03-28T18:04:22.331 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ceph/group.list 2024-03-28T18:04:22.390 INFO:teuthology.orchestra.run.smithi062.stdout:Building cifs 2024-03-28T18:04:22.407 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/cifs/group.list 2024-03-28T18:04:22.456 INFO:teuthology.orchestra.run.smithi062.stdout:Building ext4 2024-03-28T18:04:22.471 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ext4/group.list 2024-03-28T18:04:22.615 INFO:teuthology.orchestra.run.smithi062.stdout:Building f2fs 2024-03-28T18:04:22.631 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/f2fs/group.list 2024-03-28T18:04:22.696 INFO:teuthology.orchestra.run.smithi062.stdout:Building generic 2024-03-28T18:04:22.713 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/generic/group.list 2024-03-28T18:04:23.084 INFO:teuthology.orchestra.run.smithi062.stdout:Building nfs 2024-03-28T18:04:23.098 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/nfs/group.list 2024-03-28T18:04:23.151 INFO:teuthology.orchestra.run.smithi062.stdout:Building ocfs2 2024-03-28T18:04:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ocfs2/group.list 2024-03-28T18:04:23.213 INFO:teuthology.orchestra.run.smithi062.stdout:Building overlay 2024-03-28T18:04:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/overlay/group.list 2024-03-28T18:04:23.394 INFO:teuthology.orchestra.run.smithi062.stdout:Building perf 2024-03-28T18:04:23.409 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/perf/group.list 2024-03-28T18:04:23.446 INFO:teuthology.orchestra.run.smithi062.stdout:Building selftest 2024-03-28T18:04:23.457 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/selftest/group.list 2024-03-28T18:04:23.494 INFO:teuthology.orchestra.run.smithi062.stdout:Building shared 2024-03-28T18:04:23.509 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/shared/group.list 2024-03-28T18:04:23.578 INFO:teuthology.orchestra.run.smithi062.stdout:Building tmpfs 2024-03-28T18:04:23.591 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/tmpfs/group.list 2024-03-28T18:04:23.637 INFO:teuthology.orchestra.run.smithi062.stdout:Building udf 2024-03-28T18:04:23.651 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/udf/group.list 2024-03-28T18:04:23.689 INFO:teuthology.orchestra.run.smithi062.stdout:Building xfs 2024-03-28T18:04:23.704 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/xfs/group.list 2024-03-28T18:04:24.058 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo make install) 2024-03-28T18:04:24.192 INFO:teuthology.orchestra.run.smithi062.stdout:Building include 2024-03-28T18:04:24.203 INFO:teuthology.orchestra.run.smithi062.stdout:Building lib 2024-03-28T18:04:24.214 INFO:teuthology.orchestra.run.smithi062.stdout:Building ltp 2024-03-28T18:04:24.227 INFO:teuthology.orchestra.run.smithi062.stdout:Building src 2024-03-28T18:04:24.332 INFO:teuthology.orchestra.run.smithi062.stdout:Building log-writes 2024-03-28T18:04:24.344 INFO:teuthology.orchestra.run.smithi062.stdout:Building perf 2024-03-28T18:04:24.355 INFO:teuthology.orchestra.run.smithi062.stdout:Building aio-dio-regress 2024-03-28T18:04:24.367 INFO:teuthology.orchestra.run.smithi062.stdout:Building vfs 2024-03-28T18:04:24.379 INFO:teuthology.orchestra.run.smithi062.stdout:Building m4 2024-03-28T18:04:24.391 INFO:teuthology.orchestra.run.smithi062.stdout:Building common 2024-03-28T18:04:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:Building tests 2024-03-28T18:04:24.417 INFO:teuthology.orchestra.run.smithi062.stdout:Building btrfs 2024-03-28T18:04:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/btrfs/group.list 2024-03-28T18:04:24.688 INFO:teuthology.orchestra.run.smithi062.stdout:Building ceph 2024-03-28T18:04:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ceph/group.list 2024-03-28T18:04:24.762 INFO:teuthology.orchestra.run.smithi062.stdout:Building cifs 2024-03-28T18:04:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/cifs/group.list 2024-03-28T18:04:24.823 INFO:teuthology.orchestra.run.smithi062.stdout:Building ext4 2024-03-28T18:04:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ext4/group.list 2024-03-28T18:04:24.980 INFO:teuthology.orchestra.run.smithi062.stdout:Building f2fs 2024-03-28T18:04:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/f2fs/group.list 2024-03-28T18:04:25.056 INFO:teuthology.orchestra.run.smithi062.stdout:Building generic 2024-03-28T18:04:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/generic/group.list 2024-03-28T18:04:25.476 INFO:teuthology.orchestra.run.smithi062.stdout:Building nfs 2024-03-28T18:04:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/nfs/group.list 2024-03-28T18:04:25.548 INFO:teuthology.orchestra.run.smithi062.stdout:Building ocfs2 2024-03-28T18:04:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ocfs2/group.list 2024-03-28T18:04:25.612 INFO:teuthology.orchestra.run.smithi062.stdout:Building overlay 2024-03-28T18:04:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/overlay/group.list 2024-03-28T18:04:25.768 INFO:teuthology.orchestra.run.smithi062.stdout:Building perf 2024-03-28T18:04:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/perf/group.list 2024-03-28T18:04:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Building selftest 2024-03-28T18:04:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/selftest/group.list 2024-03-28T18:04:25.882 INFO:teuthology.orchestra.run.smithi062.stdout:Building shared 2024-03-28T18:04:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/shared/group.list 2024-03-28T18:04:25.960 INFO:teuthology.orchestra.run.smithi062.stdout:Building tmpfs 2024-03-28T18:04:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/tmpfs/group.list 2024-03-28T18:04:26.025 INFO:teuthology.orchestra.run.smithi062.stdout:Building udf 2024-03-28T18:04:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/udf/group.list 2024-03-28T18:04:26.083 INFO:teuthology.orchestra.run.smithi062.stdout:Building xfs 2024-03-28T18:04:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/xfs/group.list 2024-03-28T18:04:26.505 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-03-28T18:04:26.508 INFO:teuthology.orchestra.run.smithi062.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-28T18:04:26.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-03-28T18:04:26.514 INFO:teuthology.orchestra.run.smithi062.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-28T18:04:26.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-03-28T18:04:26.519 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-03-28T18:04:26.552 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-03-28T18:04:26.799 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-03-28T18:04:26.833 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-03-28T18:04:26.878 INFO:teuthology.orchestra.run.smithi062.stdout:Building log-writes 2024-03-28T18:04:26.889 INFO:teuthology.orchestra.run.smithi062.stdout:Building perf 2024-03-28T18:04:26.899 INFO:teuthology.orchestra.run.smithi062.stdout:Building aio-dio-regress 2024-03-28T18:04:26.911 INFO:teuthology.orchestra.run.smithi062.stdout:Building vfs 2024-03-28T18:04:26.919 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake -C log-writes install 2024-03-28T18:04:26.923 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-03-28T18:04:26.955 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-03-28T18:04:26.989 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake -C perf install 2024-03-28T18:04:26.993 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-03-28T18:04:27.026 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-03-28T18:04:27.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-03-28T18:04:27.115 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-03-28T18:04:27.147 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-03-28T18:04:27.335 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake -C vfs install 2024-03-28T18:04:27.339 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-03-28T18:04:27.372 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-03-28T18:04:27.416 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-03-28T18:04:27.449 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-03-28T18:04:31.800 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-03-28T18:04:32.126 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-03-28T18:04:32.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-03-28T18:04:32.216 INFO:teuthology.orchestra.run.smithi062.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-28T18:04:32.216 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-03-28T18:04:32.219 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-03-28T18:04:32.251 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-03-28T18:04:32.808 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-03-28T18:04:32.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/btrfs/ install 2024-03-28T18:04:32.819 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/btrfs/group.list 2024-03-28T18:04:33.042 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-03-28T18:04:33.074 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 /var/lib/xfstests/tests/btrfs 2024-03-28T18:04:36.697 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-03-28T18:04:36.726 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out /var/lib/xfstests/tests/btrfs 2024-03-28T18:04:40.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ceph/ install 2024-03-28T18:04:40.263 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ceph/group.list 2024-03-28T18:04:40.312 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-03-28T18:04:40.346 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-03-28T18:04:40.418 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-03-28T18:04:40.449 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-03-28T18:04:40.525 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/cifs/ install 2024-03-28T18:04:40.532 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/cifs/group.list 2024-03-28T18:04:40.575 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-03-28T18:04:40.609 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-03-28T18:04:40.638 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-03-28T18:04:40.667 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-03-28T18:04:40.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ext4/ install 2024-03-28T18:04:40.703 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ext4/group.list 2024-03-28T18:04:40.854 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-03-28T18:04:40.887 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-03-28T18:04:41.661 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-03-28T18:04:41.695 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-03-28T18:04:42.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/f2fs/ install 2024-03-28T18:04:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/f2fs/group.list 2024-03-28T18:04:42.545 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-03-28T18:04:42.578 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-03-28T18:04:42.619 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-03-28T18:04:42.658 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-03-28T18:04:42.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/generic/ install 2024-03-28T18:04:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/generic/group.list 2024-03-28T18:04:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-03-28T18:04:43.123 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-03-28T18:04:45.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:04:45.658 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:04:51.518 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-03-28T18:04:51.557 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-03-28T18:04:59.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/nfs/ install 2024-03-28T18:04:59.789 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/nfs/group.list 2024-03-28T18:04:59.839 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-03-28T18:04:59.872 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-03-28T18:04:59.907 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-03-28T18:04:59.941 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-03-28T18:04:59.974 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ocfs2/ install 2024-03-28T18:04:59.981 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/ocfs2/group.list 2024-03-28T18:05:00.029 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-03-28T18:05:00.060 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-03-28T18:05:00.094 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-03-28T18:05:00.126 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-03-28T18:05:00.159 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/overlay/ install 2024-03-28T18:05:00.165 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/overlay/group.list 2024-03-28T18:05:00.312 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-03-28T18:05:00.345 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-03-28T18:05:01.482 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-03-28T18:05:01.516 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-03-28T18:05:02.768 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/perf/ install 2024-03-28T18:05:02.775 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/perf/group.list 2024-03-28T18:05:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-03-28T18:05:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-03-28T18:05:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-03-28T18:05:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-03-28T18:05:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/selftest/ install 2024-03-28T18:05:02.956 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/selftest/group.list 2024-03-28T18:05:02.999 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-03-28T18:05:03.032 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-03-28T18:05:03.114 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-03-28T18:05:03.148 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-03-28T18:05:03.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/shared/ install 2024-03-28T18:05:03.242 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/shared/group.list 2024-03-28T18:05:03.308 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-03-28T18:05:03.338 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-03-28T18:05:03.384 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-03-28T18:05:03.415 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-03-28T18:05:03.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/tmpfs/ install 2024-03-28T18:05:03.478 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/tmpfs/group.list 2024-03-28T18:05:03.525 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-03-28T18:05:03.557 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-03-28T18:05:03.590 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-03-28T18:05:03.624 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-03-28T18:05:03.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/udf/ install 2024-03-28T18:05:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/udf/group.list 2024-03-28T18:05:03.769 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-03-28T18:05:03.769 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-03-28T18:05:03.776 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-03-28T18:05:03.810 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-03-28T18:05:03.840 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/xfs/ install 2024-03-28T18:05:03.847 INFO:teuthology.orchestra.run.smithi062.stdout: [GROUP] /tmp/tmp.7I1MPLSTiQxfstests-dev/tests/xfs/group.list 2024-03-28T18:05:04.241 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-03-28T18:05:04.278 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-03-28T18:05:12.232 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-03-28T18:05:12.268 INFO:teuthology.orchestra.run.smithi062.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-03-28T18:05:15.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:05:15.788 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:05:20.086 INFO:teuthology.orchestra.run.smithi062.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-03-28T18:05:20.119 INFO:teuthology.orchestra.run.smithi062.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-03-28T18:05:20.153 INFO:teuthology.orchestra.run.smithi062.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-03-28T18:05:20.186 INFO:teuthology.orchestra.run.smithi062.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-03-28T18:05:20.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:05:20.223 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec ls tests/generic) 2024-03-28T18:05:20.278 INFO:teuthology.orchestra.run.smithi062.stdout:001 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:001.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:002 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:002.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:003 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:003.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:004 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:004.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:005 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:005.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:006 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:006.out 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:007 2024-03-28T18:05:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:007.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:008 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:008.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:009 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:009.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:010 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:010.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:011 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:011.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:012 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:012.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:013 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:013.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:014 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:014.out 2024-03-28T18:05:20.280 INFO:teuthology.orchestra.run.smithi062.stdout:015 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:015.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:016 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:016.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:017 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:017.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:018 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:018.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:019 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:019.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:020 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:020.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:021 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:021.out 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:022 2024-03-28T18:05:20.281 INFO:teuthology.orchestra.run.smithi062.stdout:022.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:023 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:023.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:024 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:024.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:025 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:025.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:026 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:026.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:027 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:027.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:028 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:028.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:029 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:029.out 2024-03-28T18:05:20.282 INFO:teuthology.orchestra.run.smithi062.stdout:030 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:030.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:031 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:031.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:032 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:032.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:033 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:033.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:034 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:034.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:035 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:035.cfg 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:035.out.default 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:035.out.nfs 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:036 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:036.out 2024-03-28T18:05:20.283 INFO:teuthology.orchestra.run.smithi062.stdout:037 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:037.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:038 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:038.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:039 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:039.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:040 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:040.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:041 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:041.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:042 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:042.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:043 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:043.out 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:044 2024-03-28T18:05:20.284 INFO:teuthology.orchestra.run.smithi062.stdout:044.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:045 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:045.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:046 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:046.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:047 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:047.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:048 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:048.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:049 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:049.out 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:050 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:050.cfg 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:050.out.default 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:050.out.nojournal 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:050.out.xfsquota 2024-03-28T18:05:20.285 INFO:teuthology.orchestra.run.smithi062.stdout:051 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:051.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:052 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:052.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:053 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:053.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:054 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:054.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:055 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:055.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:056 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:056.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:057 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:057.out 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:058 2024-03-28T18:05:20.286 INFO:teuthology.orchestra.run.smithi062.stdout:058.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:059 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:059.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:060 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:060.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:061 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:061.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:062 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:062.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:063 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:063.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:064 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:064.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:065 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:065.out 2024-03-28T18:05:20.287 INFO:teuthology.orchestra.run.smithi062.stdout:066 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:066.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:067 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:067.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:068 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:068.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:069 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:069.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:070 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:070.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:071 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:071.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:072 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:072.out 2024-03-28T18:05:20.288 INFO:teuthology.orchestra.run.smithi062.stdout:073 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:073.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:074 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:074.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:075 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:075.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:076 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:076.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:077 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:077.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:078 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:078.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:079 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:079.out 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:080 2024-03-28T18:05:20.289 INFO:teuthology.orchestra.run.smithi062.stdout:080.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:081 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:081.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:082 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:082.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:083 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:083.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:084 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:084.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:085 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:085.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:086 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:086.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:087 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:087.out 2024-03-28T18:05:20.290 INFO:teuthology.orchestra.run.smithi062.stdout:088 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:088.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:089 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:089.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:090 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:090.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:091 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:091.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:092 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:092.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:093 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:093.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:094 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:094.out 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:095 2024-03-28T18:05:20.291 INFO:teuthology.orchestra.run.smithi062.stdout:095.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:096 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:096.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:097 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:097.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:098 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:098.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:099 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:099.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:100 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:100.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:101 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:101.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:102 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:102.out 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:103 2024-03-28T18:05:20.292 INFO:teuthology.orchestra.run.smithi062.stdout:103.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:104 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:104.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:105 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:105.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:106 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:106.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:107 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:107.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:108 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:108.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:109 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:109.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:110 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:110.out 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:111 2024-03-28T18:05:20.293 INFO:teuthology.orchestra.run.smithi062.stdout:111.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:112 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:112.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:113 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:113.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:114 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:114.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:115 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:115.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:116 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:116.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:117 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:117.out 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:118 2024-03-28T18:05:20.294 INFO:teuthology.orchestra.run.smithi062.stdout:118.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:119 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:119.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:120 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:120.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:121 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:121.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:122 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:122.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:123 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:123.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:124 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:124.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:125 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:125.out 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:126 2024-03-28T18:05:20.295 INFO:teuthology.orchestra.run.smithi062.stdout:126.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:127 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:127.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:128 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:128.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:129 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:129.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:130 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:130.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:131 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:131.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:132 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:132.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:133 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:133.out 2024-03-28T18:05:20.296 INFO:teuthology.orchestra.run.smithi062.stdout:134 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:134.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:135 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:135.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:136 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:136.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:137 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:137.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:138 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:138.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:139 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:139.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:140 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:140.out 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:141 2024-03-28T18:05:20.297 INFO:teuthology.orchestra.run.smithi062.stdout:141.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:142 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:142.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:143 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:143.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:144 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:144.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:145 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:145.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:146 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:146.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:147 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:147.out 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:148 2024-03-28T18:05:20.298 INFO:teuthology.orchestra.run.smithi062.stdout:148.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:149 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:149.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:150 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:150.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:151 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:151.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:152 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:152.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:153 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:153.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:154 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:154.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:155 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:155.out 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:156 2024-03-28T18:05:20.299 INFO:teuthology.orchestra.run.smithi062.stdout:156.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:157 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:157.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:158 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:158.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:159 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:159.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:160 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:160.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:161 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:161.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:162 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:162.out 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:163 2024-03-28T18:05:20.300 INFO:teuthology.orchestra.run.smithi062.stdout:163.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:164 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:164.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:165 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:165.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:166 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:166.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:167 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:167.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:168 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:168.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:169 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:169.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:170 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:170.out 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:171 2024-03-28T18:05:20.301 INFO:teuthology.orchestra.run.smithi062.stdout:171.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:172 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:172.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:173 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:173.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:174 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:174.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:175 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:175.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:176 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:176.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:177 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:177.out 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:178 2024-03-28T18:05:20.302 INFO:teuthology.orchestra.run.smithi062.stdout:178.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:179 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:179.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:180 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:180.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:181 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:181.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:182 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:182.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:183 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:183.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:184 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:184.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:185 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:185.out 2024-03-28T18:05:20.303 INFO:teuthology.orchestra.run.smithi062.stdout:186 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:186.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:187 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:187.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:188 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:188.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:189 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:189.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:190 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:190.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:191 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:191.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:192 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:192.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:193 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:193.out 2024-03-28T18:05:20.304 INFO:teuthology.orchestra.run.smithi062.stdout:194 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:194.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:195 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:195.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:196 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:196.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:197 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:197.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:198 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:198.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:199 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:199.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:200 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:200.out 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:201 2024-03-28T18:05:20.305 INFO:teuthology.orchestra.run.smithi062.stdout:201.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:202 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:202.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:203 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:203.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:204 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:204.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:205 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:205.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:206 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:206.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:207 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:207.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:208 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:208.out 2024-03-28T18:05:20.306 INFO:teuthology.orchestra.run.smithi062.stdout:209 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:209.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:210 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:210.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:211 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:211.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:212 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:212.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:213 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:213.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:214 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:214.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:215 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:215.out 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:216 2024-03-28T18:05:20.307 INFO:teuthology.orchestra.run.smithi062.stdout:216.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:217 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:217.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:218 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:218.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:219 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:219.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:220 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:220.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:221 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:221.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:222 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:222.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:223 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:223.out 2024-03-28T18:05:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:224 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:224.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:225 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:225.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:226 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:226.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:227 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:227.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:228 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:228.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:229 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:229.out 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:230 2024-03-28T18:05:20.309 INFO:teuthology.orchestra.run.smithi062.stdout:230.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:231 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:231.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:232 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:232.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:233 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:233.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:234 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:234.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:235 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:235.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:236 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:236.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:237 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:237.out 2024-03-28T18:05:20.310 INFO:teuthology.orchestra.run.smithi062.stdout:238 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:238.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:239 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:239.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:240 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:240.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:241 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:241.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:242 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:242.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:243 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:243.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:244 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:244.out 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:245 2024-03-28T18:05:20.311 INFO:teuthology.orchestra.run.smithi062.stdout:245.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:246 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:246.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:247 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:247.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:248 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:248.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:249 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:249.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:250 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:250.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:251 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:251.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:252 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:252.out 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:253 2024-03-28T18:05:20.312 INFO:teuthology.orchestra.run.smithi062.stdout:253.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:254 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:254.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:255 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:255.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:256 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:256.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:257 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:257.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:258 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:258.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:259 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:259.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:260 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:260.out 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:261 2024-03-28T18:05:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:261.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:262 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:262.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:263 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:263.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:264 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:264.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:265 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:265.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:266 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:266.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:267 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:267.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:268 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:268.out 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:269 2024-03-28T18:05:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:269.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:270 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:270.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:271 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:271.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:272 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:272.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:273 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:273.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:274 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:274.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:275 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:275.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:276 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:276.out 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:277 2024-03-28T18:05:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:277.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:278 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:278.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:279 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:279.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:280 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:280.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:281 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:281.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:282 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:282.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:283 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:283.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:284 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:284.out 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:285 2024-03-28T18:05:20.316 INFO:teuthology.orchestra.run.smithi062.stdout:285.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:286 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:286.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:287 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:287.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:288 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:288.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:289 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:289.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:290 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:290.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:291 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:291.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:292 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:292.out 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:293 2024-03-28T18:05:20.317 INFO:teuthology.orchestra.run.smithi062.stdout:293.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:294 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:294.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:295 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:295.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:296 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:296.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:297 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:297.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:298 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:298.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:299 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:299.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:300 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:300.out 2024-03-28T18:05:20.318 INFO:teuthology.orchestra.run.smithi062.stdout:301 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:301.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:302 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:302.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:303 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:303.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:304 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:304.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:305 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:305.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:306 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:306.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:307 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:307.out 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:308 2024-03-28T18:05:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:308.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:309 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:309.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:310 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:310.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:311 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:311.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:312 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:312.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:313 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:313.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:314 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:314.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:315 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:315.out 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:316 2024-03-28T18:05:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:316.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:317 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:317.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:318 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:318.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:319 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:319.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:320 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:320.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:321 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:321.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:322 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:322.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:323 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:323.out 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:324 2024-03-28T18:05:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:324.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:325 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:325.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:326 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:326.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:327 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:327.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:328 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:328.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:329 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:329.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:330 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:330.out 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:331 2024-03-28T18:05:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:331.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:332 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:332.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:333 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:333.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:334 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:334.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:335 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:335.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:336 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:336.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:337 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:337.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:338 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:338.out 2024-03-28T18:05:20.323 INFO:teuthology.orchestra.run.smithi062.stdout:339 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:339.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:340 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:340.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:341 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:341.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:342 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:342.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:343 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:343.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:344 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:344.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:345 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:345.out 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:346 2024-03-28T18:05:20.324 INFO:teuthology.orchestra.run.smithi062.stdout:346.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:347 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:347.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:348 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:348.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:349 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:349.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:350 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:350.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:351 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:351.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:352 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:352.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:353 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:353.out 2024-03-28T18:05:20.325 INFO:teuthology.orchestra.run.smithi062.stdout:354 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:354.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:355 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:355.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:356 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:356.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:357 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:357.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:358 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:358.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:359 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:359.out 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:360 2024-03-28T18:05:20.326 INFO:teuthology.orchestra.run.smithi062.stdout:360.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:361 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:361.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:362 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:362.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:363 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:363.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:364 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:364.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:365 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:365.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:366 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:366.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:367 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:367.out 2024-03-28T18:05:20.327 INFO:teuthology.orchestra.run.smithi062.stdout:368 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:368.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:369 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:369.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:370 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:370.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:371 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:371.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:372 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:372.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:373 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:373.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:374 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:374.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:375 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:375.out 2024-03-28T18:05:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:376 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:376.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:377 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:377.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:378 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:378.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:379 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:379.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:380 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:380.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:381 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:381.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:382 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:382.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:383 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:383.out 2024-03-28T18:05:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:384 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:384.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:385 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:385.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:386 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:386.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:387 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:387.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:388 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:388.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:389 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:389.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:390 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:390.out 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:391 2024-03-28T18:05:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:391.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:392 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:392.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:393 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:393.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:394 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:394.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:395 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:395.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:396 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:396.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:397 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:397.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:398 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:398.out 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:399 2024-03-28T18:05:20.331 INFO:teuthology.orchestra.run.smithi062.stdout:399.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:400 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:400.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:401 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:401.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:402 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:402.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:403 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:403.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:404 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:404.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:405 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:405.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:406 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:406.out 2024-03-28T18:05:20.332 INFO:teuthology.orchestra.run.smithi062.stdout:407 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:407.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:408 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:408.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:409 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:409.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:410 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:410.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:411 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:411.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:412 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:412.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:413 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:413.out 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:414 2024-03-28T18:05:20.333 INFO:teuthology.orchestra.run.smithi062.stdout:414.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:415 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:415.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:416 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:416.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:417 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:417.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:418 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:418.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:419 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:419.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:420 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:420.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:421 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:421.out 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:422 2024-03-28T18:05:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:422.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:423 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:423.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:424 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:424.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:425 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:425.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:426 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:426.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:427 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:427.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:428 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:428.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:429 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:429.out 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:430 2024-03-28T18:05:20.335 INFO:teuthology.orchestra.run.smithi062.stdout:430.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:431 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:431.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:432 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:432.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:433 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:433.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:434 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:434.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:435 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:435.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:436 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:436.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:437 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:437.out 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:438 2024-03-28T18:05:20.336 INFO:teuthology.orchestra.run.smithi062.stdout:438.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:439 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:439.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:440 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:440.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:441 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:441.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:442 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:442.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:443 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:443.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:444 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:444.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:445 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:445.out 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:446 2024-03-28T18:05:20.337 INFO:teuthology.orchestra.run.smithi062.stdout:446.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:447 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:447.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:448 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:448.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:449 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:449.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:450 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:450.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:451 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:451.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:452 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:452.out 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:453 2024-03-28T18:05:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:453.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:454 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:454.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:455 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:455.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:456 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:456.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:457 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:457.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:458 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:458.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:459 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:459.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:460 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:460.out 2024-03-28T18:05:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:461 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:461.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:462 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:462.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:463 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:463.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:464 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:464.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:465 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:465.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:466 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:466.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:467 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:467.out 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:468 2024-03-28T18:05:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:468.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:469 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:469.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:470 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:470.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:471 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:471.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:472 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:472.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:473 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:473.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:474 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:474.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:475 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:475.out 2024-03-28T18:05:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:476 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:476.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:477 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:477.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:478 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:478.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:479 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:479.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:480 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:480.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:481 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:481.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:482 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:482.out 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:483 2024-03-28T18:05:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:483.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:484 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:484.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:485 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:485.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:486 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:486.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:487 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:487.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:488 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:488.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:489 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:489.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:490 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:490.out 2024-03-28T18:05:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:491 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:491.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:492 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:492.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:493 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:493.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:494 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:494.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:495 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:495.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:496 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:496.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:497 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:497.out 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:498 2024-03-28T18:05:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:498.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:499 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:499.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:500 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:500.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:501 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:501.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:502 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:502.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:503 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:503.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:504 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:504.out 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:505 2024-03-28T18:05:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:505.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:506 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:506.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:507 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:507.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:508 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:508.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:509 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:509.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:510 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:510.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:511 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:511.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:512 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:512.out 2024-03-28T18:05:20.346 INFO:teuthology.orchestra.run.smithi062.stdout:513 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:513.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:514 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:514.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:515 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:515.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:516 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:516.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:517 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:517.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:518 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:518.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:519 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:519.out 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:520 2024-03-28T18:05:20.347 INFO:teuthology.orchestra.run.smithi062.stdout:520.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:521 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:521.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:522 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:522.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:523 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:523.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:524 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:524.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:525 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:525.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:526 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:526.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:527 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:527.out 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:528 2024-03-28T18:05:20.348 INFO:teuthology.orchestra.run.smithi062.stdout:528.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:529 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:529.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:530 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:530.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:531 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:531.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:532 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:532.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:533 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:533.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:534 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:534.out 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:535 2024-03-28T18:05:20.349 INFO:teuthology.orchestra.run.smithi062.stdout:535.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:536 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:536.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:537 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:537.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:538 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:538.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:539 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:539.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:540 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:540.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:541 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:541.out 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:542 2024-03-28T18:05:20.350 INFO:teuthology.orchestra.run.smithi062.stdout:542.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:543 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:543.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:544 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:544.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:545 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:545.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:546 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:546.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:547 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:547.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:548 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:548.out 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:549 2024-03-28T18:05:20.351 INFO:teuthology.orchestra.run.smithi062.stdout:549.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:550 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:550.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:551 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:551.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:552 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:552.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:553 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:553.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:554 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:554.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:555 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:555.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:556 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:556.out 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:557 2024-03-28T18:05:20.352 INFO:teuthology.orchestra.run.smithi062.stdout:557.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:558 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:558.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:559 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:559.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:560 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:560.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:561 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:561.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:562 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:562.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:563 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:563.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:564 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:564.out 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:565 2024-03-28T18:05:20.353 INFO:teuthology.orchestra.run.smithi062.stdout:565.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:566 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:566.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:567 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:567.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:568 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:568.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:569 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:569.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:570 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:570.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:571 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:571.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:572 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:572.out 2024-03-28T18:05:20.354 INFO:teuthology.orchestra.run.smithi062.stdout:573 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:573.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:574 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:574.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:575 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:575.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:576 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:576.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:577 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:577.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:578 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:578.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:579 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:579.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:580 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:580.out 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:581 2024-03-28T18:05:20.355 INFO:teuthology.orchestra.run.smithi062.stdout:581.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:582 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:582.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:583 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:583.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:584 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:584.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:585 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:585.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:586 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:586.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:587 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:587.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:588 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:588.out 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:589 2024-03-28T18:05:20.356 INFO:teuthology.orchestra.run.smithi062.stdout:589.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:590 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:590.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:591 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:591.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:592 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:592.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:593 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:593.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:594 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:594.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:595 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:595.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:596 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:596.out 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:597 2024-03-28T18:05:20.357 INFO:teuthology.orchestra.run.smithi062.stdout:597.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:598 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:598.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:599 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:599.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:600 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:600.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:601 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:601.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:602 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:602.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:603 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:603.out 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:604 2024-03-28T18:05:20.358 INFO:teuthology.orchestra.run.smithi062.stdout:604.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:605 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:605.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:606 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:606.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:607 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:607.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:608 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:608.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:609 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:609.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:610 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:610.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:611 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:611.out 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:612 2024-03-28T18:05:20.359 INFO:teuthology.orchestra.run.smithi062.stdout:612.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:613 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:613.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:614 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:614.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:615 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:615.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:616 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:616.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:617 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:617.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:618 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:618.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:619 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:619.out 2024-03-28T18:05:20.360 INFO:teuthology.orchestra.run.smithi062.stdout:620 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:620.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:621 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:621.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:622 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:622.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:623 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:623.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:624 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:624.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:625 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:625.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:626 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:626.out 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:627 2024-03-28T18:05:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:627.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:628 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:628.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:629 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:629.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:630 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:630.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:631 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:631.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:632 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:632.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:633 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:633.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:634 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:634.out 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:635 2024-03-28T18:05:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:635.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:636 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:636.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:637 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:637.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:638 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:638.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:639 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:639.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:640 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:640.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:641 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:641.out 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:642 2024-03-28T18:05:20.363 INFO:teuthology.orchestra.run.smithi062.stdout:642.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:643 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:643.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:644 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:644.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:645 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:645.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:646 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:646.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:647 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:647.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:648 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:648.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:649 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:649.out 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:650 2024-03-28T18:05:20.364 INFO:teuthology.orchestra.run.smithi062.stdout:650.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:651 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:651.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:652 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:652.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:653 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:653.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:654 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:654.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:655 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:655.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:656 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:656.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:657 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:657.out 2024-03-28T18:05:20.365 INFO:teuthology.orchestra.run.smithi062.stdout:658 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:658.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:659 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:659.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:660 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:660.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:661 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:661.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:662 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:662.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:663 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:663.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:664 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:664.out 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:665 2024-03-28T18:05:20.366 INFO:teuthology.orchestra.run.smithi062.stdout:665.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:666 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:666.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:667 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:667.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:668 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:668.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:669 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:669.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:670 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:670.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:671 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:671.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:672 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:672.out 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:673 2024-03-28T18:05:20.367 INFO:teuthology.orchestra.run.smithi062.stdout:673.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:674 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:674.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:675 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:675.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:676 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:676.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:677 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:677.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:678 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:678.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:679 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:679.out 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:680 2024-03-28T18:05:20.368 INFO:teuthology.orchestra.run.smithi062.stdout:680.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:681 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:681.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:682 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:682.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:683 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:683.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:684 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:684.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:685 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:685.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:686 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:686.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:687 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:687.out 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:688 2024-03-28T18:05:20.369 INFO:teuthology.orchestra.run.smithi062.stdout:688.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:689 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:689.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:690 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:690.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:691 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:691.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:692 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:692.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:693 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:693.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:694 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:694.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:695 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:695.out 2024-03-28T18:05:20.370 INFO:teuthology.orchestra.run.smithi062.stdout:696 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:696.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:697 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:697.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:698 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:698.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:699 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:699.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:700 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:700.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:701 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:701.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:702 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:702.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:703 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:703.out 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:704 2024-03-28T18:05:20.371 INFO:teuthology.orchestra.run.smithi062.stdout:704.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:705 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:705.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:706 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:706.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:707 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:707.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:708 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:708.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:709 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:709.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:710 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:710.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:711 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:711.out 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:712 2024-03-28T18:05:20.372 INFO:teuthology.orchestra.run.smithi062.stdout:712.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:713 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:713.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:714 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:714.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:715 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:715.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:716 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:716.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:717 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:717.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:718 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:718.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:719 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:719.out 2024-03-28T18:05:20.373 INFO:teuthology.orchestra.run.smithi062.stdout:720 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:720.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:721 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:721.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:722 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:722.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:723 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:723.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:724 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:724.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:725 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:725.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:726 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:726.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:727 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:727.out 2024-03-28T18:05:20.374 INFO:teuthology.orchestra.run.smithi062.stdout:728 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:728.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:729 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:729.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:730 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:730.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:731 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:731.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:732 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:732.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:733 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:733.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:734 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:734.out 2024-03-28T18:05:20.375 INFO:teuthology.orchestra.run.smithi062.stdout:735 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:735.out 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:736 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:736.out 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:737 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:737.out 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:738 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:738.out 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:739 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:739.out 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:group.list 2024-03-28T18:05:20.376 INFO:teuthology.orchestra.run.smithi062.stdout:Makefile 2024-03-28T18:05:20.377 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-03-28T18:05:20.884 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:20.898 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:20.898 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:29.219 INFO:teuthology.orchestra.run.smithi062.stdout:generic/001 9s 2024-03-28T18:05:29.237 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/001 2024-03-28T18:05:29.237 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:05:29.246 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:29.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[50812]: fuse finished with error 0 and tester_r 0 2024-03-28T18:05:29.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:05:29.326 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-03-28T18:05:29.832 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:29.846 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:29.847 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:31.089 INFO:teuthology.orchestra.run.smithi062.stdout:generic/002 2s 2024-03-28T18:05:31.104 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/002 2024-03-28T18:05:31.104 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:05:31.113 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:31.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:05:31.148 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-03-28T18:05:31.625 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:31.637 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:31.637 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:32.038 INFO:teuthology.orchestra.run.smithi062.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:05:32.048 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/003 2024-03-28T18:05:32.048 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/003 2024-03-28T18:05:32.048 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:05:32.059 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:32.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:05:32.087 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-03-28T18:05:32.588 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:32.601 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:33.049 INFO:teuthology.orchestra.run.smithi062.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-03-28T18:05:33.060 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/004 2024-03-28T18:05:33.060 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/004 2024-03-28T18:05:33.060 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:05:33.068 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:33.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:05:33.092 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-03-28T18:05:33.631 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:33.642 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:33.642 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:38.027 INFO:teuthology.orchestra.run.smithi062.stdout:generic/005 5s 2024-03-28T18:05:38.043 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/005 2024-03-28T18:05:38.043 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:05:38.052 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:38.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:05:38.096 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-03-28T18:05:38.570 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:05:38.582 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:05:38.582 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:05:45.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:05:45.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:06:16.058 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:06:16.066 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:06:46.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:06:46.209 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:07:16.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:07:16.370 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:07:46.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:07:46.502 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:08:16.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:08:16.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:08:46.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:08:46.764 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:09:02.249 INFO:teuthology.orchestra.run.smithi062.stdout:generic/006 204s 2024-03-28T18:09:02.262 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/006 2024-03-28T18:09:02.262 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:09:02.269 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:09:02.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:09:02.311 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-03-28T18:09:02.783 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:09:02.794 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:09:02.794 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:09:16.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:09:16.906 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:09:47.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:09:47.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:10:17.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:10:17.257 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:10:47.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:10:47.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:11:17.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:11:17.538 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:11:47.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:11:47.680 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:12:17.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:12:17.828 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:12:47.974 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:12:47.984 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:13:18.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:13:18.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:13:48.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:13:48.253 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:14:18.392 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:14:18.400 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:14:49.353 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:14:49.363 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:15:19.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:15:19.514 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:15:49.644 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:15:49.654 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:16:19.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:16:19.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:16:49.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:16:49.934 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:17:20.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:17:20.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:17:50.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:17:50.245 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:18:03.015 INFO:teuthology.orchestra.run.smithi062.stdout:generic/007 541s 2024-03-28T18:18:03.031 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/007 2024-03-28T18:18:03.032 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:18:03.041 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:03.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:18:03.080 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-03-28T18:18:03.578 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:18:03.589 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:18:03.589 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:06.783 INFO:teuthology.orchestra.run.smithi062.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-28T18:18:06.798 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/008 2024-03-28T18:18:06.799 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/008 2024-03-28T18:18:06.799 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:18:06.809 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:06.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:18:06.843 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-03-28T18:18:07.380 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:18:07.393 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:18:07.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:10.742 INFO:teuthology.orchestra.run.smithi062.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-28T18:18:10.754 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/009 2024-03-28T18:18:10.754 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/009 2024-03-28T18:18:10.754 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:18:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:10.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:18:10.789 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-03-28T18:18:11.314 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:18:11.325 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:18:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:13.248 INFO:teuthology.orchestra.run.smithi062.stdout:generic/010 2s 2024-03-28T18:18:14.633 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/010 2024-03-28T18:18:14.633 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:18:14.644 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:14.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:18:14.683 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-03-28T18:18:15.177 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:18:15.189 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:18:15.190 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:18:20.435 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:18:20.445 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:18:50.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:18:50.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:19:20.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:19:20.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:19:21.384 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-03-28T18:19:21.384+0000 7f714ee00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 114967) UID: 0 2024-03-28T18:19:21.385 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-03-28T18:19:21.384+0000 7fbb8ca00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 114967) UID: 0 2024-03-28T18:20:26.514 INFO:teuthology.orchestra.run.smithi062.stdout:generic/011 131s 2024-03-28T18:20:26.819 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/011 2024-03-28T18:20:26.819 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:20:26.832 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:20:26.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:20:26.872 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-03-28T18:20:27.358 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:20:27.369 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:20:27.369 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:20:30.000 INFO:teuthology.orchestra.run.smithi062.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:20:30.011 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/012 2024-03-28T18:20:30.011 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/012 2024-03-28T18:20:30.011 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:20:30.019 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:20:30.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:20:30.059 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-03-28T18:20:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:20:30.544 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:20:30.544 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:22:42.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:22:42.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:23:12.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:23:12.334 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:23:22.364 INFO:teuthology.orchestra.run.smithi062.stdout:generic/013 172s 2024-03-28T18:23:22.384 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/013 2024-03-28T18:23:22.385 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:22.396 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:22.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:22.432 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-03-28T18:23:23.126 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:23.144 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:23.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:23.713 INFO:teuthology.orchestra.run.smithi062.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-03-28T18:23:23.726 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/014 2024-03-28T18:23:23.727 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/014 2024-03-28T18:23:23.727 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:23.738 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:23.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:23.779 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-03-28T18:23:24.463 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:24.478 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:24.938 INFO:teuthology.orchestra.run.smithi062.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:23:24.951 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/015 2024-03-28T18:23:24.951 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/015 2024-03-28T18:23:24.952 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:24.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:24.997 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-03-28T18:23:25.674 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:25.690 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:29.215 INFO:teuthology.orchestra.run.smithi062.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:23:29.228 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/016 2024-03-28T18:23:29.228 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/016 2024-03-28T18:23:29.228 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:29.239 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:29.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:29.274 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-03-28T18:23:30.033 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:30.049 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:30.049 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:30.486 INFO:teuthology.orchestra.run.smithi062.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:23:30.500 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/017 2024-03-28T18:23:30.500 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/017 2024-03-28T18:23:30.500 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:30.515 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:30.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:30.551 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-03-28T18:23:31.238 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:31.254 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:31.255 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:31.691 INFO:teuthology.orchestra.run.smithi062.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:23:31.704 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/018 2024-03-28T18:23:31.705 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/018 2024-03-28T18:23:31.705 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:31.715 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:31.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:31.751 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-03-28T18:23:32.427 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:32.450 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:32.450 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi062.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:23:32.913 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/019 2024-03-28T18:23:32.913 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/019 2024-03-28T18:23:32.913 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:23:32.922 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:32.958 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:23:32.958 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-03-28T18:23:33.636 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:33.650 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:33.650 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:42.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:23:42.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:23:58.491 INFO:teuthology.orchestra.run.smithi062.stdout:generic/020 - output mismatch (see /tmp/tmp.7I1MPLSTiQxfstests-dev/results//generic/020.out.bad) 2024-03-28T18:23:58.501 INFO:teuthology.orchestra.run.smithi062.stdout: --- tests/generic/020.out 2024-03-28 18:00:34.244577705 +0000 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +++ /tmp/tmp.7I1MPLSTiQxfstests-dev/results//generic/020.out.bad 2024-03-28 18:23:58.485972286 +0000 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: @@ -47,9 +47,13 @@ 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: user.snrub="fish2\012" 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: *** really long value 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: -0000000 00 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: -* 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: -ATTRSIZE 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +attr_set: No space left on device 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +Could not set "long_attr" for 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +attr_get: No data available 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +Could not get "long_attr" for 2024-03-28T18:23:58.502 INFO:teuthology.orchestra.run.smithi062.stdout: +0000000 2024-03-28T18:23:58.503 INFO:teuthology.orchestra.run.smithi062.stdout: +attr_remove: No data available 2024-03-28T18:23:58.503 INFO:teuthology.orchestra.run.smithi062.stdout: +Could not remove "long_attr" for 2024-03-28T18:23:58.503 INFO:teuthology.orchestra.run.smithi062.stdout: *** set/get/remove really long names (expect failure) 2024-03-28T18:23:58.503 INFO:teuthology.orchestra.run.smithi062.stdout: attr_set: Invalid argument 2024-03-28T18:23:58.503 INFO:teuthology.orchestra.run.smithi062.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-03-28T18:23:58.514 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/020 2024-03-28T18:23:58.514 INFO:teuthology.orchestra.run.smithi062.stdout:Failures: generic/020 2024-03-28T18:23:58.514 INFO:teuthology.orchestra.run.smithi062.stdout:Failed 1 of 1 tests 2024-03-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:23:58.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T18:23:58.590 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-03-28T18:23:58.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-03-28T18:23:58.591 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-03-28T18:23:59.350 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:23:59.367 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:23:59.367 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:02.691 INFO:teuthology.orchestra.run.smithi062.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:24:02.707 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/021 2024-03-28T18:24:02.707 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/021 2024-03-28T18:24:02.707 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:02.717 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:02.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:02.761 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-03-28T18:24:03.534 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:03.548 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:03.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:24:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/022 2024-03-28T18:24:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/022 2024-03-28T18:24:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:07.051 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:07.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:07.103 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-03-28T18:24:07.880 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:07.898 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:07.898 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:12.511 INFO:teuthology.orchestra.run.smithi062.stdout:generic/023 5s 2024-03-28T18:24:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/023 2024-03-28T18:24:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:12.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:12.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:12.579 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-03-28T18:24:12.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:24:12.686 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:24:13.394 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:13.410 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:13.410 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:13.939 INFO:teuthology.orchestra.run.smithi062.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-03-28T18:24:13.951 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/024 2024-03-28T18:24:13.952 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/024 2024-03-28T18:24:13.952 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:13.961 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:14.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:14.006 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-03-28T18:24:14.792 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:14.809 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:15.391 INFO:teuthology.orchestra.run.smithi062.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-03-28T18:24:15.406 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/025 2024-03-28T18:24:15.406 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/025 2024-03-28T18:24:15.406 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:15.417 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:15.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:15.463 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-03-28T18:24:16.259 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:16.277 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:16.859 INFO:teuthology.orchestra.run.smithi062.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-03-28T18:24:16.872 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/026 2024-03-28T18:24:16.872 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/026 2024-03-28T18:24:16.872 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:16.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:16.921 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-03-28T18:24:17.626 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:17.643 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:18.012 INFO:teuthology.orchestra.run.smithi062.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:18.025 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/027 2024-03-28T18:24:18.025 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/027 2024-03-28T18:24:18.025 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:18.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:18.068 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-03-28T18:24:18.755 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:18.774 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:18.774 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:24.283 INFO:teuthology.orchestra.run.smithi062.stdout:generic/028 6s 2024-03-28T18:24:24.302 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/028 2024-03-28T18:24:24.303 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:24.314 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:24.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:24.352 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-03-28T18:24:25.023 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:25.042 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:25.496 INFO:teuthology.orchestra.run.smithi062.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:25.510 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/029 2024-03-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/029 2024-03-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:25.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:25.558 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-03-28T18:24:26.271 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:26.288 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:26.756 INFO:teuthology.orchestra.run.smithi062.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:26.769 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/030 2024-03-28T18:24:26.770 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/030 2024-03-28T18:24:26.770 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:26.779 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:26.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:26.812 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-03-28T18:24:27.485 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:27.504 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:27.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:27.956 INFO:teuthology.orchestra.run.smithi062.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:27.970 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/031 2024-03-28T18:24:27.971 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/031 2024-03-28T18:24:27.971 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:27.980 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:28.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:28.016 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-03-28T18:24:28.706 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:28.721 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:28.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:29.159 INFO:teuthology.orchestra.run.smithi062.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:29.178 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/032 2024-03-28T18:24:29.178 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/032 2024-03-28T18:24:29.178 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:29.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:29.223 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-03-28T18:24:29.891 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:29.906 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:29.906 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:30.375 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/033 2024-03-28T18:24:30.375 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/033 2024-03-28T18:24:30.376 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:30.387 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:30.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:30.419 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-03-28T18:24:31.099 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:31.115 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:31.116 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:31.600 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/034 2024-03-28T18:24:31.601 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/034 2024-03-28T18:24:31.601 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:31.613 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:31.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:31.645 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-03-28T18:24:32.339 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:32.358 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:32.358 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:33.043 INFO:teuthology.orchestra.run.smithi062.stdout:generic/035 1s 2024-03-28T18:24:33.064 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/035 2024-03-28T18:24:33.066 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:33.074 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:33.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:33.122 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-03-28T18:24:33.892 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:33.912 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:33.912 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:42.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:24:42.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:24:44.567 INFO:teuthology.orchestra.run.smithi062.stdout:generic/036 11s 2024-03-28T18:24:44.589 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/036 2024-03-28T18:24:44.589 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:44.599 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:44.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:44.652 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-03-28T18:24:45.409 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:45.426 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:45.426 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:45.863 INFO:teuthology.orchestra.run.smithi062.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:45.877 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/037 2024-03-28T18:24:45.878 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/037 2024-03-28T18:24:45.878 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:45.888 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:45.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:45.917 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-03-28T18:24:46.600 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:46.617 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:46.617 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:47.056 INFO:teuthology.orchestra.run.smithi062.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:47.067 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/038 2024-03-28T18:24:47.067 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/038 2024-03-28T18:24:47.067 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:47.081 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:47.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:47.114 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-03-28T18:24:47.780 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:47.795 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:47.795 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:48.266 INFO:teuthology.orchestra.run.smithi062.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:48.279 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/039 2024-03-28T18:24:48.280 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/039 2024-03-28T18:24:48.280 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:48.291 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:48.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:48.328 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-03-28T18:24:49.006 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:49.021 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:49.021 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:49.500 INFO:teuthology.orchestra.run.smithi062.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:49.514 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/040 2024-03-28T18:24:49.514 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/040 2024-03-28T18:24:49.515 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:49.528 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:49.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:49.565 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-03-28T18:24:50.263 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:50.278 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:50.279 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:50.774 INFO:teuthology.orchestra.run.smithi062.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:50.789 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/041 2024-03-28T18:24:50.789 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/041 2024-03-28T18:24:50.789 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:50.800 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:50.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:50.832 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-03-28T18:24:51.509 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:51.525 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:51.525 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:51.967 INFO:teuthology.orchestra.run.smithi062.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:51.980 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/042 2024-03-28T18:24:51.980 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/042 2024-03-28T18:24:51.980 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:51.990 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:52.021 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:52.022 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-03-28T18:24:52.683 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:52.698 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:53.104 INFO:teuthology.orchestra.run.smithi062.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:53.113 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/043 2024-03-28T18:24:53.113 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/043 2024-03-28T18:24:53.113 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:53.121 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:53.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:53.142 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-03-28T18:24:53.727 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:53.745 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:53.745 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:54.210 INFO:teuthology.orchestra.run.smithi062.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:54.223 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/044 2024-03-28T18:24:54.223 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/044 2024-03-28T18:24:54.223 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:54.232 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:54.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:54.267 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-03-28T18:24:54.956 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:54.970 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:54.970 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:55.414 INFO:teuthology.orchestra.run.smithi062.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:55.426 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/045 2024-03-28T18:24:55.426 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/045 2024-03-28T18:24:55.426 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:55.435 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:55.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:55.472 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-03-28T18:24:56.150 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:56.167 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:56.168 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:56.617 INFO:teuthology.orchestra.run.smithi062.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:56.672 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/046 2024-03-28T18:24:56.672 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/046 2024-03-28T18:24:56.672 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:56.672 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:56.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:56.680 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-03-28T18:24:57.360 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:57.375 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:57.375 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:57.808 INFO:teuthology.orchestra.run.smithi062.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:57.821 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/047 2024-03-28T18:24:57.822 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/047 2024-03-28T18:24:57.822 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:57.832 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:57.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:57.864 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-03-28T18:24:58.521 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:58.537 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:58.537 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:58.969 INFO:teuthology.orchestra.run.smithi062.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:24:58.981 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/048 2024-03-28T18:24:58.982 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/048 2024-03-28T18:24:58.982 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:24:58.991 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:24:59.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:24:59.021 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-03-28T18:24:59.680 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:24:59.697 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:24:59.697 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:00.135 INFO:teuthology.orchestra.run.smithi062.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:00.148 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/049 2024-03-28T18:25:00.148 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/049 2024-03-28T18:25:00.149 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:00.159 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:00.197 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:00.197 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-03-28T18:25:00.898 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:00.915 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:00.915 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:01.363 INFO:teuthology.orchestra.run.smithi062.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:01.376 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/050 2024-03-28T18:25:01.376 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/050 2024-03-28T18:25:01.376 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:01.386 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:01.421 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:01.422 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-03-28T18:25:02.094 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:02.111 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:02.567 INFO:teuthology.orchestra.run.smithi062.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:02.581 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/051 2024-03-28T18:25:02.581 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/051 2024-03-28T18:25:02.581 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:02.593 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:02.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:02.628 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-03-28T18:25:03.293 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:03.312 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:03.313 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:03.775 INFO:teuthology.orchestra.run.smithi062.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:03.790 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/052 2024-03-28T18:25:03.792 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/052 2024-03-28T18:25:03.792 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:03.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:03.836 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-03-28T18:25:04.508 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:04.525 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:04.525 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:04.983 INFO:teuthology.orchestra.run.smithi062.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/053 2024-03-28T18:25:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/053 2024-03-28T18:25:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:05.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:05.047 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-03-28T18:25:05.730 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:06.197 INFO:teuthology.orchestra.run.smithi062.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:06.209 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/054 2024-03-28T18:25:06.209 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/054 2024-03-28T18:25:06.209 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:06.218 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:06.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:06.253 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-03-28T18:25:06.938 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:06.958 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:06.958 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:07.434 INFO:teuthology.orchestra.run.smithi062.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:07.448 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/055 2024-03-28T18:25:07.448 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/055 2024-03-28T18:25:07.448 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:07.461 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:07.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:07.496 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-03-28T18:25:08.198 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:08.218 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:08.219 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:08.720 INFO:teuthology.orchestra.run.smithi062.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:08.734 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/056 2024-03-28T18:25:08.734 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/056 2024-03-28T18:25:08.734 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:08.748 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:08.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:08.785 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-03-28T18:25:09.446 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:09.465 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:09.465 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:09.933 INFO:teuthology.orchestra.run.smithi062.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:09.945 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/057 2024-03-28T18:25:09.945 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/057 2024-03-28T18:25:09.946 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:09.954 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:09.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:09.993 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-03-28T18:25:10.691 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:10.705 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:13.061 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:25:13.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:25:14.029 INFO:teuthology.orchestra.run.smithi062.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:25:14.044 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/058 2024-03-28T18:25:14.044 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/058 2024-03-28T18:25:14.045 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:14.057 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:14.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:14.110 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-03-28T18:25:14.891 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:14.906 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:14.907 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:15.389 INFO:teuthology.orchestra.run.smithi062.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:15.401 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/059 2024-03-28T18:25:15.401 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/059 2024-03-28T18:25:15.401 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:15.412 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:15.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:15.444 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-03-28T18:25:16.107 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:16.125 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:19.460 INFO:teuthology.orchestra.run.smithi062.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:25:19.473 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/060 2024-03-28T18:25:19.473 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/060 2024-03-28T18:25:19.473 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:19.481 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:19.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:19.546 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-03-28T18:25:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:20.334 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:20.335 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:23.656 INFO:teuthology.orchestra.run.smithi062.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:25:23.671 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/061 2024-03-28T18:25:23.671 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/061 2024-03-28T18:25:23.671 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:23.681 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:23.730 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:23.731 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-03-28T18:25:24.524 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:24.541 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:25.003 INFO:teuthology.orchestra.run.smithi062.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:25.018 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/062 2024-03-28T18:25:25.019 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/062 2024-03-28T18:25:25.019 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:25.063 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:25.063 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-03-28T18:25:25.736 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:25.749 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:28.938 INFO:teuthology.orchestra.run.smithi062.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-28T18:25:28.949 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/063 2024-03-28T18:25:28.949 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/063 2024-03-28T18:25:28.949 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:28.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:29.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:29.007 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-03-28T18:25:29.828 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:29.846 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:29.847 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:30.307 INFO:teuthology.orchestra.run.smithi062.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:30.321 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/064 2024-03-28T18:25:30.321 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/064 2024-03-28T18:25:30.321 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:30.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:30.372 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-03-28T18:25:31.051 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:31.067 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:31.067 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:31.543 INFO:teuthology.orchestra.run.smithi062.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:31.559 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/065 2024-03-28T18:25:31.559 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/065 2024-03-28T18:25:31.559 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:31.573 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:31.607 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:31.608 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-03-28T18:25:32.264 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:32.284 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:32.284 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:32.752 INFO:teuthology.orchestra.run.smithi062.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:32.764 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/066 2024-03-28T18:25:32.764 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/066 2024-03-28T18:25:32.764 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:32.773 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:32.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:32.807 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-03-28T18:25:33.481 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:33.498 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:33.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:34.016 INFO:teuthology.orchestra.run.smithi062.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:34.029 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/067 2024-03-28T18:25:34.030 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/067 2024-03-28T18:25:34.030 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:34.042 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:34.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:34.080 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-03-28T18:25:34.741 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:34.757 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:34.757 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:35.749 INFO:teuthology.orchestra.run.smithi062.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:35.761 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/068 2024-03-28T18:25:35.761 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/068 2024-03-28T18:25:35.761 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:35.768 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:35.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:35.801 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-03-28T18:25:36.493 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:36.513 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:36.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:36.977 INFO:teuthology.orchestra.run.smithi062.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:25:36.991 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/069 2024-03-28T18:25:36.991 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/069 2024-03-28T18:25:36.991 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:25:37.001 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:37.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:25:37.039 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-03-28T18:25:37.699 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:25:37.721 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:25:37.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:25:43.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:25:43.236 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:26:13.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:26:13.396 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:26:43.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:26:43.611 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:27:13.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:27:13.816 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:27:43.982 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:27:43.989 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:27:49.873 INFO:teuthology.orchestra.run.smithi062.stdout:generic/070 132s 2024-03-28T18:27:49.895 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/070 2024-03-28T18:27:49.896 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:27:49.906 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:49.950 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:27:49.951 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-03-28T18:27:50.758 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:27:50.773 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:27:50.773 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:51.202 INFO:teuthology.orchestra.run.smithi062.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:27:51.218 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/071 2024-03-28T18:27:51.218 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/071 2024-03-28T18:27:51.218 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:27:51.230 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:51.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:27:51.274 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-03-28T18:27:51.934 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:27:51.948 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:27:51.948 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:54.855 INFO:teuthology.orchestra.run.smithi062.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-03-28T18:27:54.870 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/072 2024-03-28T18:27:54.870 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/072 2024-03-28T18:27:54.870 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:27:54.883 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:54.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:27:54.918 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-03-28T18:27:55.562 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:56.058 INFO:teuthology.orchestra.run.smithi062.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/073 2024-03-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/073 2024-03-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:27:56.083 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:27:56.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:27:56.115 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-03-28T18:27:56.771 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:27:56.789 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:27:56.789 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:28:14.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:28:14.125 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:28:44.311 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:28:44.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:29:14.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:29:14.500 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:29:44.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:29:44.943 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:30:15.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:30:15.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:30:45.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:30:45.330 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:31:15.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:31:15.490 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:31:45.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:31:45.687 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:32:15.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:32:15.875 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:32:46.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:32:46.013 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:33:16.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:33:16.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:33:46.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:33:46.419 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:34:16.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:34:16.610 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:34:46.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:34:46.790 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:35:16.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:35:16.944 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:35:47.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:35:47.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:36:17.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:36:17.258 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:36:47.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:36:47.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:37:18.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:37:18.097 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:37:49.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:37:49.592 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:38:19.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:38:19.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:38:49.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:38:49.874 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:39:20.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:39:20.036 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:39:50.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:39:50.202 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:40:21.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:40:21.559 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:40:51.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:40:51.695 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:41:21.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:41:21.841 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:41:52.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:41:52.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:42:22.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:42:22.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:42:52.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:42:52.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:43:22.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:43:22.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:43:52.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:43:52.984 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:44:23.132 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:44:23.138 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:44:53.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:44:53.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:45:23.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:45:23.439 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:45:53.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:45:53.579 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:46:23.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:46:23.748 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:46:53.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:46:53.897 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:47:24.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:47:24.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:47:54.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:47:54.187 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:48:24.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:48:24.370 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:48:54.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:48:54.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:49:24.678 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:49:24.685 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:49:54.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:49:54.822 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:50:24.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:50:24.975 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:50:56.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:50:56.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:51:28.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:51:28.967 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:51:59.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:51:59.144 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:52:29.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:52:29.322 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:52:59.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:52:59.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:53:30.014 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:53:30.023 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:54:00.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:54:00.201 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:54:30.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:54:30.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:55:00.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:55:00.543 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:55:30.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:55:30.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:56:01.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:56:01.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:56:31.700 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:56:31.708 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:57:01.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:57:01.869 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:57:31.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:57:31.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:58:02.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:58:02.123 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:58:12.480 INFO:teuthology.orchestra.run.smithi062.stdout:generic/074 1816s 2024-03-28T18:58:12.512 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/074 2024-03-28T18:58:12.512 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T18:58:12.521 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:58:12.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T18:58:12.558 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-03-28T18:58:13.613 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T18:58:13.625 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T18:58:13.625 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T18:58:32.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:58:32.257 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:59:02.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:59:02.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:59:32.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T18:59:32.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:00:02.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:00:02.675 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:00:14.835 INFO:teuthology.orchestra.run.smithi062.stdout:generic/075 121s 2024-03-28T19:00:14.850 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/075 2024-03-28T19:00:14.850 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:14.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:14.898 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-03-28T19:00:15.639 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:15.652 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:15.653 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:16.063 INFO:teuthology.orchestra.run.smithi062.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:16.077 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/076 2024-03-28T19:00:16.077 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/076 2024-03-28T19:00:16.077 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:16.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:16.118 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-03-28T19:00:16.666 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:16.678 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:17.911 INFO:teuthology.orchestra.run.smithi062.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:17.926 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/077 2024-03-28T19:00:17.926 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/077 2024-03-28T19:00:17.926 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:17.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:17.972 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-03-28T19:00:18.473 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:18.486 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:18.486 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:19.001 INFO:teuthology.orchestra.run.smithi062.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-03-28T19:00:19.010 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/078 2024-03-28T19:00:19.011 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/078 2024-03-28T19:00:19.011 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:19.020 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:19.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:19.073 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-03-28T19:00:19.787 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:19.800 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:19.800 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:20.308 INFO:teuthology.orchestra.run.smithi062.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-03-28T19:00:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/079 2024-03-28T19:00:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/079 2024-03-28T19:00:20.321 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:20.331 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:20.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:20.362 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-03-28T19:00:20.922 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:20.935 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:20.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:23.515 INFO:teuthology.orchestra.run.smithi062.stdout:generic/080 3s 2024-03-28T19:00:23.532 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/080 2024-03-28T19:00:23.532 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:23.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:23.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:23.573 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-03-28T19:00:24.098 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:24.111 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:24.112 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:24.553 INFO:teuthology.orchestra.run.smithi062.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:24.567 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/081 2024-03-28T19:00:24.567 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/081 2024-03-28T19:00:24.567 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:24.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:24.610 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-03-28T19:00:25.133 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:25.145 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:25.575 INFO:teuthology.orchestra.run.smithi062.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:25.587 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/082 2024-03-28T19:00:25.587 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/082 2024-03-28T19:00:25.587 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:25.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:25.626 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-03-28T19:00:26.158 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:26.170 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:26.610 INFO:teuthology.orchestra.run.smithi062.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:26.623 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/083 2024-03-28T19:00:26.623 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/083 2024-03-28T19:00:26.623 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:26.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:26.663 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-03-28T19:00:27.225 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:27.237 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:27.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:27.617 INFO:teuthology.orchestra.run.smithi062.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:27.626 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/084 2024-03-28T19:00:27.627 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/084 2024-03-28T19:00:27.627 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:27.635 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:27.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:27.664 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-03-28T19:00:28.197 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:28.209 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:28.209 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:28.654 INFO:teuthology.orchestra.run.smithi062.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-03-28T19:00:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/085 2024-03-28T19:00:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Not run: generic/085 2024-03-28T19:00:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:28.677 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:28.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:28.710 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-03-28T19:00:29.289 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:29.302 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:29.302 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:32.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:00:32.822 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:00:51.451 INFO:teuthology.orchestra.run.smithi062.stdout:generic/086 22s 2024-03-28T19:00:51.514 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/086 2024-03-28T19:00:51.514 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:51.529 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:51.681 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:51.682 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-03-28T19:00:53.453 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:53.478 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:53.478 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:54.472 INFO:teuthology.orchestra.run.smithi062.stdout:generic/087 1s 2024-03-28T19:00:54.488 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/087 2024-03-28T19:00:54.488 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:54.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:54.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:54.535 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-03-28T19:00:55.269 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:55.281 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:55.838 INFO:teuthology.orchestra.run.smithi062.stdout:generic/088 0s 2024-03-28T19:00:55.854 INFO:teuthology.orchestra.run.smithi062.stdout:Ran: generic/088 2024-03-28T19:00:55.854 INFO:teuthology.orchestra.run.smithi062.stdout:Passed all 1 tests 2024-03-28T19:00:55.863 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:00:55.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-28T19:00:55.906 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/tmp.7I1MPLSTiQxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-03-28T19:00:56.611 INFO:teuthology.orchestra.run.smithi062.stdout:FSTYP -- ceph-fuse 2024-03-28T19:00:56.623 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM -- Linux/x86_64 smithi062 6.8.0-g1df0c7e8a235 #1 SMP PREEMPT_DYNAMIC Mon Mar 25 03:32:35 UTC 2024 2024-03-28T19:00:56.623 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-28T19:01:02.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:01:02.980 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:01:33.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:01:33.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:02:03.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:02:03.278 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:02:33.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:02:33.427 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:03:03.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:03:03.576 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:03:35.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:03:35.656 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:04:05.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:04:05.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:04:35.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:04:35.931 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:05:06.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:05:06.044 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:05:36.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:05:36.180 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:06:06.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:06:06.299 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:06:06.408 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-03-28T19:06:06.409+0000 7f714ee00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 157715) UID: 0 2024-03-28T19:06:06.409 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-03-28T19:06:06.409+0000 7fbb8ca00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 157715) UID: 0 2024-03-28T19:09:28.157 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:09:28.169 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:09:58.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:09:58.310 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:10:28.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:10:28.442 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:10:58.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:10:58.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:11:28.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:11:28.744 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:11:58.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:11:58.894 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:12:29.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:12:29.043 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:12:59.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:12:59.182 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:13:29.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:13:29.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:13:59.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:13:59.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:14:30.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:14:30.070 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:15:00.197 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:15:00.205 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:15:00.304 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T19:15:00.304+0000 7fab91200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157942) UID: 0 2024-03-28T19:15:00.305 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T19:15:00.304+0000 7faa33000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157942) UID: 0 2024-03-28T19:15:00.305 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T19:15:00.305+0000 7fb88b000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157942) UID: 0 2024-03-28T19:15:00.305 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T19:15:00.305+0000 7fd795600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 157942) UID: 0 2024-03-28T19:18:17.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:18:17.330 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:18:17.790 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-03-28T19:18:17.791+0000 7fbb8ca00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 158174) UID: 0 2024-03-28T19:18:17.790 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-03-28T19:18:17.791+0000 7f714ee00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 158174) UID: 0 2024-03-28T19:21:26.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:21:26.227 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:21:56.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:21:56.345 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:22:26.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:22:26.475 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:22:57.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:22:57.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:23:28.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:23:28.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:23:58.263 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:23:58.272 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:23:58.368 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-03-28T19:23:58.369+0000 7fab91200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158291) UID: 0 2024-03-28T19:23:58.369 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-03-28T19:23:58.369+0000 7faa33000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158291) UID: 0 2024-03-28T19:23:58.369 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-03-28T19:23:58.369+0000 7fb88b000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158291) UID: 0 2024-03-28T19:23:58.369 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-03-28T19:23:58.369+0000 7fd795600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158291) UID: 0 2024-03-28T19:23:58.399 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-03-28T19:23:58.399+0000 7fcd54c00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46676) UID: 0 2024-03-28T19:23:58.399 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-03-28T19:23:58.399+0000 7f2a04200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46676) UID: 0 2024-03-28T19:23:58.399 INFO:tasks.ceph.osd.6.smithi081.stderr:2024-03-28T19:23:58.399+0000 7f7848600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46676) UID: 0 2024-03-28T19:23:58.399 INFO:tasks.ceph.osd.7.smithi081.stderr:2024-03-28T19:23:58.399+0000 7fb6e4800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46676) UID: 0 2024-03-28T19:27:04.599 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:27:04.607 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:27:34.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:27:34.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:28:04.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:28:04.881 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:28:35.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:28:35.023 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T19:28:51.848 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-28T19:28:51.848 DEBUG:teuthology.task.console_log:Killing console logger for smithi062 2024-03-28T19:28:51.849 DEBUG:teuthology.task.console_log:Killing console logger for smithi081 2024-03-28T19:28:51.849 DEBUG:teuthology.exit:Finished running handlers